In-Ear Utility Device Having Voice Recognition

ABSTRACT

An embodiment of the invention provides a wireless in-ear utility device that rests in the user&#39;s ear canal near the user&#39;s eardrum. The in-ear utility device may be configured in a variety of ways, including, but in no way limited to a smart in-ear utility device, a flexible personal sound amplification product, a personal music player, a “walkie-talkie” and the like.

CROSS-REFERENCE TO RELATED APPLICATIONS

The present application is related to co-filed U.S. patent applicationSer. No. ______ entitled “In-Ear Utility Device Having Tap Detector,”(Attorney Docket Number SmartEar-002US); U.S. patent application Ser.No. ______ entitled “In-Ear Utility Device Having Dual Microphones,”(Attorney Docket Number SmartEar-006US); U.S. patent application Ser.No. ______ entitled “In-Ear Utility Device Having Sensors,” (AttorneyDocket Number SmartEar-007US); U.S. patent application Ser. No. ______entitled “In-Ear Utility Device Having A Humidity Sensor,” (AttorneyDocket Number SmartEar-008US); U.S. patent application Ser. No. ______entitled “In-Ear Utility Device Having Information Sharing,” (AttorneyDocket Number SmartEar-009US), and U.S. patent application Ser. No.______ entitled “In-Ear Utility Device Having Voice Recognition”(Attorney Docket Number SmartEar-010US), which are assigned to theassignee of the present application. These related applications areincorporated herein by reference in their entirety.

FIELD

Embodiments of the invention relate to systems and methods pertaining toin-ear utility devices. More particularly, an embodiment of theinvention relates to systems and methods that employ in-ear electronicsto provide a wireless in-ear utility device that rests in the user's earcanal.

BACKGROUND

The following description includes information that may be useful inunderstanding embodiments of the invention. It is not an admission thatany of the information provided herein is prior art or relevant to thepresently claimed invention, or that any publication specifically orimplicitly referenced is prior art.

With the development of portable multimedia devices and smart phones,many types of ear pieces, such as earphones and headsets, have beendeveloped and used. However, previous ear pieces have traditionally beenbulky and uncomfortable as well as being limited in their technologicalabilities. Thus, the prospects for exploring new form factors for earpieces have conventionally been limited.

Moreover, these ear pieces have conventionally been devices slaved toother devices, such as smartphones, with limited abilities to operateindependently. Similarly, the prospects for exploring new andindependent uses for ear pieces have also been limited conventionally.

Therefore, a need exists for more advanced in-ear utility devices thatcan perform an expanded set of tasks at an improved rate of performanceover the devices found in the prior art.

SUMMARY OF THE INVENTION

Embodiments of the invention provide a wireless in-ear utility devicethat comprises a body having at least a portion shaped to fit into auser's ear canal, the body having a proximal end configured to reside inthe user's ear canal at a distance no more than 12 millimeters away fromthe user's ear drum. The wireless in-ear utility device comprises amicrophone in the body that receives external sounds from the in-earutility device, wherein the external sounds include sounds produced byan entity other than the user of the in-ear utility device. The wirelessin-ear utility device also includes a data storage component having anaudio profile for the entity. The wireless in-ear utility deviceincludes a processor configured to perform noise cancellation on theexternal sounds using the audio profile to enhance sounds produced bythe entity. The wireless in-ear utility device further comprises aspeaker located at a distal end of the body that receives the enhancedsounds produced by the entity and plays them to the user of the in-earutility device.

Embodiments of the invention comprise a method for operating wirelessin-ear utility device. The method comprises receiving external sounds ina microphone located in a body of the in-ear utility device, the bodyhaving at least a portion shaped to fit into the user's ear canal andhaving a proximal end configured to reside in the user's ear canal at adistance no more than 12 millimeters away from the user's ear drum,wherein the received external sounds include sounds produced by anentity other than the user of the in-ear utility device. The methodfurther comprises retrieving an audio profile for the entity from a datastorage component attached to the body. The method includes performingnoise cancellation on the external sounds by a processor configured toapply the audio profile to enhance sounds produced by the entity. Themethod also comprises playing the enhanced sounds produced by the entityto the user of the in-ear utility device through a speaker located at adistal end of the body.

BRIEF DESCRIPTION OF THE DRAWINGS

Figures provided herein may or may not be provided to scale. Therelative dimensions or proportions may vary. Embodiments of theinvention may be sized to fit within an ear canal of a user.

FIG. 1A illustrates an in-ear utility device 101 inserted into an ear105, according to an embodiment of the invention.

FIG. 1B illustrates a cross section taken along the line AA shown inFIG. 1A, according to an embodiment of the invention.

FIG. 2A provides a block diagram that illustrates an in-ear utilitydevice 201, according to an embodiment of the invention.

FIG. 2B provides a diagram 220 that shows a shock provided by a firsttap 221 and the shock provided by a second tap 223 as measured by anaccelerometer sensor 206 a in an in-ear sound device 201, according toan embodiment of the invention.

FIG. 2C provides a geometrical representation of tap sensing in whichthe taps are based on distance/tap events/time frame/coordinate,according to an embodiment of the invention.

FIG. 2D illustrates a first humidity sensor 261 and a second humiditysensor 266 that provides a means for turning an in-ear utility device263 on/off and/or components of the in-ear utility device 263 usingrelative changes in humidity, according to an embodiment of theinvention.

FIG. 2E illustrates a sensor array 231 that provides a means for turningthe in-ear utility device and/or components of the in-ear utility deviceon/off using relative changes in temperature and humidity, according toan embodiment of the invention.

FIG. 2F illustrates how the output from a relative humidity sensor 241and the output from a temperature sensor 243 can be provided to aprocessor, such as the processor 207 shown in FIG. 2A, according to anembodiment of the invention.

FIG. 3 illustrates an in-ear utility device 303 having a flexible seal302 that covers a portion of the in-ear utility device 303 that isinserted into a user's ear canal (e.g., the ear canal 115 shown in FIG.1A) during normal use, according to an embodiment of the invention.

FIG. 4 illustrates an in-ear utility device 401 with its deformableseals removed, according to an embodiment of the invention.

FIGS. 5A-5B illustrate an in-ear utility device 501 inserted into an earcanal 515, according to an embodiment of the invention.

FIG. 5C illustrates a portion of the distal ends of two in-ear utilitydevices 501 a, 501 b in a single user's ears 505 a, 505 b, according toan embodiment of the invention.

FIG. 5D illustrates a top-down view of in-ear sound devices 505 a, 505 bperforming binaural beamforming for sounds in front of the head 554shown in FIG. 5C, according to an embodiment of the invention.

FIG. 5E illustrates an in-ear utility device 560 having two ambientnoise microphones 562, 564 and two ambient noise microphone ports 561,563 along with a voice focused microphone 566 and a voice focusedmicrophone port 565, according to an embodiment of the invention.

FIGS. 5F-5H illustrate a 360-degree slit port 524 for the microphoneport on a distal end 522 of an in-ear utility device 520, according toan alternative embodiment of embodiment of the invention.

FIGS. 6A-6C illustrate a swivel joint 603 in the in-ear utility device601 that allows the in-ear utility device 601 to pivot from zero(vertical) to negative 30 degrees from zero to plus 30 degrees.

FIG. 7 illustrates an embodiment of an in-ear utility device 701configured to function as a headphone, according to an embodiment of theinvention.

FIG. 8 illustrates an embodiment of an in-ear utility device 801configured to function as a music player, according to an embodiment ofthe invention.

FIG. 9 illustrates an embodiment of an in-ear utility device 901configured to provide hearing amplification, according to an embodimentof the invention.

FIG. 10 illustrates an embodiment of an in-ear utility device 1001configured to provide a walkie-talkie function (a portable, two-wayradio transceiver), according to an embodiment of the invention.

FIG. 11 illustrates an embodiment of an in-ear utility device 1101configured as a single, integrated body rather than as a multi-piecedbody as shown and described in FIG. 3.

FIGS. 12A-12D illustrate a recharging case 1200 configured to recharge apair of in-ear utility devices 1201, 1202, according to an embodiment ofthe invention.

FIG. 13 illustrates a network 1300 through which various processingtasks for in-ear utility devices 1301 a, 1301 b can be distributed,according to an embodiment of the invention.

DETAILED DESCRIPTION OF AN EMBODIMENT OF THE INVENTION

Embodiments of the invention provide a wireless in-ear utility devicehaving a speaker placed closer to the user's eardrum than conventionalsound-delivery devices but not typically as close as somemedically-regulated hearing aids. Embodiments of the in-ear utilitydevice may be used for a variety of purposes and include a variety ofelectronic packages, such as for use as an amplified hearing device, foruse as a music player, for use as a headphone device, and for use invarious health-monitoring applications.

Embodiments of the invention provide a wireless in-ear utility deviceconfigured to have a variety of electronic packages. The electronicpackages may serve a variety of functions, such as a Bluetooth device, anoise cancellation device that allows the user to focus on sounds ofinterest, a health-monitoring device, and a fitness device, eachembodiment having the sensors and electronic configuration needed tocarry out its mission. Embodiments of the wireless in-ear utility devicemay include an electronic package that supports the Internet of Things(IoT), defined as a network of physical objects embedded withelectronics, software, sensors, and network connectivity, which enablesthe collection and exchange data between the in-ear utility device andother devices and/or the user. The Internet of Things allows objects tobe sensed and controlled remotely across existing networkinfrastructure, allowing more direct integration between the physicalworld and computer-based systems.

The sensors of the in-ear utility device may perform a variety offunctions. An accelerometer sensor 206 a mounted in the body 210, forexample, can be used to measure the user's steps. The ear (e.g., the ear105 shown in FIG. 1A) is a stable location from which to make step countmeasurements since the head moves less independently from the legs thanthe arms (e.g., false positive step counts associated with wrist/armborne accelerometers won't occur for people who make a lot of gestureswith their arms while not otherwise moving). The accelerometer workingwith the processor 207 and the data repository 209 can detect motionindications indicative of a step. When a received motion matches thepattern for a step, then a step counter can be incremented. Theaccelerometer sensor, as discussed below, can also provide an operatoralertness function, and the accelerometer sensor, as discussed below canalso be used to provide a user interface based on tap detection.

Embodiments of the invention may provide a smart in-ear utility devicethat offers heightened and/or enhanced sounds for a variety of uses froma personal music player to a “walkie-talkie” type personal communicator.Embodiments of the invention may also provide an in-ear utility devicethat includes a wireless communications module that employs a wirelessprotocol so that the in-ear utility device may communicate with externaldevices, such as a mobile computing device, another in-ear utilitydevice, a vehicle-borne computer, or a remote server or network, e.g., acloud.

Embodiments of the invention may further provide an in-ear “smartphone,”e.g., a smart device having functionality rivaling that of a smartphonebut using user interfaces appropriate for an aural rather than a visualdevice, including but not limited to voice recognition technology. The“smartphone” embodiment of the in-ear utility device may also (oralternatively) include a visual user interface operating on some form ofa computing platform, or a visual display device tethered to the in-earutility device, according to an embodiment of the invention. The visualuser interface does not have to comprise a “screen” but could beprovided by some form of altered reality and/or virtual reality (AR-VR),according to an embodiment of the invention. The “smartphone” embodimentof the in-ear utility device may include an audio user interface,according to an embodiment of the invention.

Electronic component packages used in embodiments of the in-ear utilitydevice may comprise, for example, micro-electronic devices. Electroniccomponents may include a microphone, an amplifier, a battery, a speaker,a wireless communications module, and/or any combination thereof. Theelectronic component package in some embodiments may include a processor(e.g., a CPU) and/or a data storage component. For example, theelectronic component package 113 may include functionality for executingany number of software applications (“apps”) and/or storing data such asmedia.

FIG. 1A illustrates an in-ear utility device 101 inserted into an ear105, according to an embodiment of the invention. The in-ear utilitydevice 101 includes an electronics package 113, such as the electronicscomponent package 202 shown in FIG. 2A. Embodiments of the in-earutility device 101 may include a speaker 108 disposed at the proximaltip 107 of the body of the in-ear utility device 101 and a microphone110, disposed in the distal portion 111 of the in-ear utility device101.

Some embodiments of the in-ear device 101 are designed to rest in theear 105 between 8 to 12 mm. away from the user's tympanic membrane(eardrum) 104. Thus, the in-ear utility device 101 when placed properlyin the ear canal 115 has a proximal tip 107 (along with the speaker 108)that lies from 8 to 12 mm. from the outer edge 106 of the eardrum 104along a longitudinal axis 109, according to an embodiment of theinvention. Studies have shown that the length of the typical human earcanal 115 varies from 27 mm to 35 mm measured along a curved centeraxis. Thus, embodiments of the in-ear utility device 101 reside wellinside the ear canal 115.

The distance of the in-ear utility device 101 to a given user's eardrum104 varies based on the depth of the user's ear canal 115. Some usershave shallow ear canals while other users have deep ear canals.Therefore, the distance of the in-ear utility device 101 may vary indepth from user to user. The in-ear utility device 101 comprises a body112 having the longitudinal axis 109 extending between a distal end 111and a proximal tip 107. The distal end 111 of the in-ear utility device101 resides just outside the user's ear 105 so that the in-ear utilitydevice 101 may be easily removed by hand, according to an embodiment ofthe invention. In some embodiments of the invention, the in-ear utilitydevice 101 might reside inside the ear canal 115 with no part of thedevice outside the ear 105.

In some embodiments, the speaker 108 may contact the eardrum 104 or bein even closer proximity to the eardrum than indicated in FIG. 1A, e.g.,with the possible assistance of an audiologist. (The assistance of anaudiologist is not normally needed for proper operation of the in-earutility device 201.) In some embodiments of the invention, the in-earutility device may reside in a broader range than 8 to 12 mm. from theuser's eardrum 104, e.g., 3 mm. to 15 mm. The 8 to 12 mm. range,however, should provide improved sound quality to the user while alsoresiding at a distance that does not require the employment of anaudiologist to satisfy health and safety regulations.

In contrast with the in-ear utility device 101, conventional earbuds fitin the outer ear and face but are not inserted in the user's ear canal.Similarly, conventional in-ear headphones (e.g., in-ear monitors orIEMs) are inserted in the ear canal 115 but at a considerable distance(e.g., no closer than 20 mm away from the typical user's eardrum 104).Thus, the in-ear utility device 101 resides closer to the user's eardrum104 than conventional headphones, earbuds, and in-ear headphones. Amongother things, having the in-ear utility device 101 inserted in theuser's ear canal 115 will aid in keeping the in-ear utility device 101attached to the user even when the user is engaged in physicallystrenuous activities.

FIG. 1B illustrates a cross section taken along the line AA shown inFIG. 1A, according to an embodiment of the invention. As shown in FIG.1B, in practical application, the in-ear utility device 101 is insertedinto the ear canal 115 where it conforms to the shape of the ear canal115. The typical use's ear canal 115 is not likely to comprise a perfectcircle as shown in FIG. 1B.

The in-ear utility device 101 does not typically fill up the whole ofthe user's ear canal 115. One or more gaps 126 may occur between thein-ear utility device 101 and the ear canal 115. The gaps 126 may lowerpressure in the user's ear canal 115.

The in-ear utility device 101 is typically covered with a seal or softtip 114, and the in-ear utility device 101 typically touches the earcanal 115 at the points where the tip or seal 114 touches the ear canal115 and possibly at the far outer entrance to the ear canal 115. Theseal 114 might not cover portions of the in-ear utility device 101outside the user's ear canal 115, according to an embodiment of theinvention.

The seal 114 is configured to create gaps 126 between the in-ear utilitydevice 101 and the ear canal 115, according to an embodiment of theinvention. These gaps 126 not only lower pressure in the ear canal 115,the gaps also serve the additional purpose of allowing ambient sounds topass through to the user's eardrum 104. Thus, a user wearing the in-earutility device 101 can continue to experience ambient sounds in anatural manner (e.g., constant sound stimulus), and the user's own voiceshould sound normal to him/her. The ability to still hear ambient soundsnaturally while wearing an ear-borne hearing device does not commonlyoccur with devices such as headphones and hearing aids. In addition, thein-ear utility device 101 not touching many points on the ear canal 115should also increase user comfort and provide better heat transfer,allowing the in-ear utility device 101 to be worn for extended periodsof time, according to an embodiment of the invention.

Studies show that the cross-sectional area in the middle portions of thetypical human ear canal 115 range between 25 mm² and 70 mm². Thus, theembodiments of the seal 114 need to cover a fairly wide range ofdiameters. Thus, the seal 114 may be available in a variety of sizes,although the body 112 may be manufactured in a single size, according toan embodiment of the invention.

The seal 114 allows the portion of the body 112 that rests in the user'sear canal 115 to be narrower than the ear canal 115. Thus, the body 112that contains the electronic package 113 does not typically touch theuser's ear canal 115. The presence of the seal 114 protects the useragainst malfunctions of the electronics package 113. So, for example, ifthe battery (e.g., the battery 213 shown in FIG. 2A) happens to developa short, the user should be protected from shock and heat because of thepresence of the seal 114. The user is protected by the seal 114 in partbecause many embodiments of the seal 114 are constructed from anon-metallic material (i.e., lower heat transfer and possiblyinsulating).

The seal 114 may also include one or more slits 128 configured torelieve pressure in the user's ear canal 115, allowing the in-earutility device 101 to be worn comfortably by the user for long periodsof time. The slits 128 also provide the user with non-occluded auralaccess to ambient sounds outside the user's ear 105, according to anembodiment of the invention.

Thus, portions of the user's ear canal 115 remain non-occluded by thein-ear utility device 101 due, in part, to the slits 128 and the gaps126. A user of the in-ear utility device 101 is typically able to hearsounds external to the in-ear utility device 101 and should also notsuffer from increased pressure in the ear canal 115 due to the presenceof the in-ear utility device 101 in the user's ear canal 115, asdiscussed above.

The material selection for the in-ear utility device 101 may facilitatethe in-ear utility device 101 in entering the ear 105 while facilitatingretention of the in-ear utility device 101 in the ear 105 for longperiods of time (e.g. while exercising). Embodiments of the inventionprovide an in-ear utility device 101 covered in (e.g., the seal 114) (orcomposed of) a deformable material that is comfortable to wear for along period of time and can be produced in bulk eliminating the need forcustomization. For example, the seal 114 covering the in-ear utilitydevice 101 may be customized to account for variations in size of user'sear canals (e.g., small, medium, and large).

The body 112 could comprise a variety of materials, including variousmetals. As noted above, the seal 114 protects the user's ear canal 115from the body 112 of the in-ear utility device 101. In addition, asshown in FIG. 11, an embodiment of the invention may be extremely small(e.g., nano sized). The seal 114 may be formed of a material that has aShore A Durometer hardness value of between 20-30. In an alternativeembodiment of the invention, the body 112 of the in-ear utility device101 itself may be formed of a material that has a Shore A Durometerhardness value of between 20-30. In such an embodiment, the body 112serves a function similar to the seal 114.

An electronic component package 113 is fixed inside, mounted on, and/orembedded in or on the body 112 of the in-ear utility device 101 andincludes electronic circuitry configured to allow the in-ear utilitydevice 101 to be inserted into the user's ear canal 115 without damagingthe in-ear utility device 101 or causing injury to the user's ear 105,according to an embodiment of the invention. The electronic componentpackage 113 includes a speaker 108 at its proximal end 107, according toan embodiment of the invention. The seal 114 reduces the size availablefor the electronic component package 113. Thus, the specific componentsin the electronic component package 113 may need to be carefullyselected for small size, in addition to other characteristics, accordingto an embodiment of the invention.

FIGS. 1A-1B illustrate an in-ear utility device 101 inserted into ahuman ear 105. Embodiments of the in-ear utility device 101 may beconfigured for non-human ears, such as other primates, other mammals,and even non-mammalian species. Components of the electronics componentpackage and the elastic body would be sized accordingly in theseembodiments of the invention.

FIG. 2A provides a block diagram that illustrates an in-ear utilitydevice 201, according to an embodiment of the invention. The in-earutility device 201 is formed of a body 210 that contains an electroniccomponent package 202. The specific configuration of the electroniccomponent package 202 may vary from embodiment to embodiment of thein-ear utility device 201, as discussed, for example, with respect toFIGS. 2B-11. In some embodiments, the in-ear utility device 201 may beon the order of about 5 mm. to 5 cm in length.

The body 210 may include a deformable seal (e.g., the seal 302 shown inFIG. 3) to allow the in-ear utility device 201 to be inserted into auser's ear canal (e.g., the ear canal 115 shown in FIG. 1A) withoutdamaging the in-ear utility device 201 or causing harm to the user's ear(e.g. the ear 105 shown in FIG. 1A). The body 210 may be made in onesize, and the covering seal (e.g. the seal 302 shown in FIG. 3) allowsthe in-ear utility device 201 to conform to a broad range of ear canalanatomies, according to an embodiment of the invention. The seal mayhave several different sizes, according to an embodiment of theinvention.

Embodiments of the in-ear utility device 201 may be waterproof and wornin many environments, such as during swimming or while bathing. Thein-ear utility device 201 may also be worn during sleep withoutdiscomfort. This may allow the in-ear utility device 201 to be utilizedduring many times when conventional sound devices may be uncomfortable,simply not work, or even be dangerous to use.

Electronic Component Package

The electronic component package 202 may include one or more electroniccomponents such as a microphone 203, a wireless communications module(e.g., transceiver) 204, an amplifier 205, a battery 213, a processor207, a speaker 208, a voice recognition chip 214, a Hall Effect sensor219, and a data storage component 209, various sensors 206 a-206 z,according to an embodiment of the invention. The electronic componentpackage 202 may include multiple copies of the same components, e.g.,two microphones, either for backup purposes or to provide expandedcapabilities. The individual components in the electronic componentpackage 202 may be electrically coupled and/or wired as needed toprovide conventional functionality for such components in a manner knownto ordinarily skilled artisans, except when noted herein.

The small form factor for the in-ear utility device 201 typicallyrequires the application of the smaller electronic components than thecomponents typically found in other head-mounted devices, such asBluetooth devices, according to an embodiment of the invention. Thecircuit connecting the electronic components suggests the application offlexible circuitry. Flexible electronics, also known as flex circuits,provide a means for assembling electronic circuits by mountingelectronic devices on flexible plastic substrates, such as polyimide,PEEK, or transparent conductive polyester film. Additionally, flexcircuits can be screen printed silver circuits on polyester. Flexibleelectronic assemblies may be manufactured using identical componentsused for more rigid printed circuit boards, allowing the board toconform to a desired shape, and/or to flex during its use.

Many types of electronic components may be employed in the in-earutility device 201, as discussed above. For example, in variousembodiments, the in-ear utility device may include microelectronics,nanoelectronics, micro-circuitry, nano-circuitry and combinationsthereof.

Microphone and Speaker

The microphone 203 may communicate with the speaker 208. The microphone203 may be in electronic and/or mechanical communication with thespeaker 208. Sound/vibrations picked up by the microphone 203 may betransmitted to the speaker 208. In some embodiments, thesound/vibrations picked up may be amplified via the amplifier 205 andtransmitted to the speaker 208. In various embodiments, the amplifier205 may include a digital signal processor (DSP) 212. In variousembodiments of the invention, the DSP 212 may perform (or assist in) anumber of functions, such as noise cancellation and speech recognition.The DSP 212 need not be co-located with the amplifier 205, according toembodiments of the invention.

The microphone 203 may be a significantly stronger microphone thantypically found in hear aid devices. For example, the microphone mayoperate in the range of 80 Hz to 5000 KHz, a range not typically foundin hearing aids. The microphone 203 at this range detects sounds at amuch lower decibel range than the typical hearing aid and essentiallydetects a whole spectrum of human hearing, according to an embodiment ofthe invention.

Because the processor 207 and the microphone 203 may be more powerfulthan similar components found in hearing aids, the in-ear utility device201 may need to remove white noise generated by the processor 207,especially given the more powerful microphone 203 while noise removalcan be accomplished by means of an appropriate audio filter.

A typical hearing aid microphone also operates at a comparatively lowvoltage such as 1.2V in comparison to the more powerful microphone 203that operates at 3.5 to 5V. Thus, the circuitry inside the in-earutility device 201 also needs to filter out white noise generated by itspowerful electrical components, according to an embodiment of theinvention.

The speaker 208 may be a significantly smaller speaker than typicallyfound in Bluetooth devices. This smaller speaker 208 in combination withthe smaller form factor of the body 210 allows the in-ear utility device201 to penetrate farther into the user's ear canal than a Bluetoothdevice.

The microphone 203 does not need to communicate with the speaker 208,exclusively, or at all in various embodiments of the invention. Themicrophone 203 may be employed for tasks not directly connected with thespeaker 208 and vice versa. In an embodiment of the invention, themicrophone 203 communicates sounds to the processor 207, the DSP 212,and/or the voice recognition chip 214, and/or other apparatus todetermine the type of environment that the user is located in (e.g.,dense urban area, barren wilderness, etc.) and allow the processor 207to make an appropriate action, depending on the task(s) set for thein-ear utility device 201.

The speaker 208 typically resides closer to the user's eardrum (e.g.,the eardrum 104 shown in FIG. 1A) than the microphone 203 duringoperation. As shown in FIG. 1A, the speaker 108 is disposed at theproximal tip 107 of the body of the in-ear utility device 101 while themicrophone 110 is disposed in the distal portion 111 of the in-earutility device 101. The microphone 203 may be external to the ear, orcloser to ear canal opening.

In some embodiments, the distance between the speaker 208 and themicrophone 203 may range between from 5 mm to 5 cm. As a general matter,the greater the distance between the microphone 203 and the speaker 208,the lower likelihood of feedback between the microphone 203 and thespeaker 208. The speaker 208 and the microphone 203 may be placed closertogether if feedback between the components can be nullified orcompensated for, according to an embodiment of the invention.

However, in some embodiments, the dimensions of the in-ear utilitydevice 201 and/or the distance between the microphone 203 and thespeaker 208 might be smaller and/or larger than the dimensions/distancesprovided above. For example, an embodiment of the invention may beprepared for users wearing helmets (e.g., as police officers, soldiers,football players, motorcyclists, and/or bicyclists). Similarly, anembodiment of the in-ear utility device 201 made for security personnel,hunters, etc. might be extended in size to accommodate additionalmicrophones, or higher fidelity microphones, and/or enhancedcommunications equipment.

In embodiments, audio input to the speaker 208 may come from thewireless communications module 204, such as when the wirelesscommunications module 204 is configured for Bluetooth® communications.Additionally, audio input to the speaker 208 may come from the datastorage component 209 of the in-ear utility device 201. For example,playing stored music or instructions. These configurations may alsoinclude inputs from the microphone 203 but could occur without amicrophone being included in the device.

Processor and Data Storage

In some embodiments, the in-ear utility device 201 includes a processor207 which may be integral with the electronic component package 202 oroperate under the control of a remote computing device (e.g., a mobilecomputing device) sending instructions via the communications module204.

The processor 207 in the in-ear utility device 201 may access dataand/or execute software applications 211, according to an embodiment ofthe invention. The data and software applications 211 may be stored inthe data storage component 209 and/or delivered to the processor 207 viathe communications module 204 from a remote storage device located awayfrom the in-ear utility device 201. For example, the processor 207 mightexecute a software application that resides on a mobile phone linked tothe in-ear utility device 201. A skilled artisan will appreciate thatmany software applications known in the art may be utilized by theprocessor 207. A variety of different data and software applicationsherein have been labeled 211, as an indication that the data and/orsoftware applications are stored in the data storage component 209.

For example, the processor 207 may be configured withprocessor-executable instructions 211 to perform operations todistinguish meaningful sound, such as speech, from ambient noise. Suchinstructions may perform operations for receiving sound signals from themicrophone 203, determining whether the sound signals representmeaningful sound, according to various criteria stored in the datastorage component 209, providing the sounds to the speaker 208 when thesound signals represent meaningful sound, and filtering the sounds fromthe speaker 208 when the sound signals do not represent meaningfulsound. Such instructions 211 for a speech detection program may bepresent in the data storage component 209 of the in-ear utility device201 or a coupled external computing device.

The processor 207 may comprise a CPU, or a like computing device, or mayalternatively comprise a simple circuit that directs the operations ofthe various components in the electronic component package 202,according to an embodiment of the invention. In embodiments in which theprocessor 207 comprises a simple control circuit, the other componentsin the electronic component package 202 may also be simple and/or few innumber, e.g., just a battery 213, a data storage component 209, and aspeaker 208, in addition to the processor 207.

In some embodiments, the processor 207 may be a significantly morepowerful computing device than conventionally found in hearing aids. Forexample, the processor 207 might be a CSR8670 chip. CSR8670 is an audiosystem-on-chip (SoC) solution with wireless connectivity, embedded flashmemory and integrated capacitive touch sensors. The CSR8670 includesnoise cancellation and voice recognition capabilities. Thus, in someembodiments of the invention, the processor 207 may include some of theother components shown in FIG. 2A. In contrast, the typicalcompletely-in-ear-canal (CIC) hearing aid (e.g., a hearing aid in theear canal rather than behind the ear) uses an SB3229-E1 chip or similarprocessing chip, which has a slower speed and a smaller feature set thanthe processor 207. The processor 207 may require higher power than thetypical hearing aid processor. The CSR8670 chip requires between 4V to2.8V. The SB3229-E1 chip operates at much lower voltage, e.g., 1.2V. TheCSR8670 chip operates at 20-34 milliamps while the SB3229-E1 chipoperates in the micro-amps range. Thus, placing the processor 207 intothe body 210 may require careful adjustment in order to operateproperly, according to an embodiment of the invention. The filtering ofwhite noise, for example, has already been mentioned.

The data storage component 209 may comprise a non-transitory memory,such as RAM, flash, ROM, hard drive, solid state, drive, optical mediaand the like. The data storage component 209 may include various typesof data, such as media, music, software, and the like. The data storagecomponent 209 may have a variety of sizes, e.g., 1 to 4 gigabytes,according to an embodiment of the invention. In-the-ear-canal (CIC)hearing aids, by comparison, typically have much smaller size memories.Integrating the data storage component 209 into the in-ear utilitydevice 201 requires care to make sure that components function properlyin the small form factor.

Wireless Communication Module

The wireless communications module 204 can be implemented using acombination of hardware (e.g., driver circuits, antennas, transceivers,modulators/demodulators, encoders/decoders, and other analog and/ordigital signal processing circuits) and software components. Multipledifferent wireless communication protocols and associated hardware canbe incorporated into the wireless communications module 204.

The wireless communications module 204 includes structural andfunctional components known in the art to facilitate wirelesscommunication with another computing device or a remote network. Thewireless communications module 204 can include RF transceiver componentssuch as an antenna and supporting circuitry to enable data communicationover a wireless medium, e.g., using Wi-Fi (IEEE 802.11 familystandards), Bluetooth® (a family of standards promulgated by BluetoothSIG, Inc.), or other protocols for wireless data communication. In someembodiments, the wireless communications module 204 can implement ashort-range sensor (e.g., Bluetooth, BLE or ultra-wide band).

In some embodiments, the wireless communications module 204 can providenear-field communication (“NFC”) capability, e.g., implementing theISO/IEC 18092 standards or the like. NFC can support wireless dataexchange between devices over a very short range (e.g., 20 centimetersor less). NFC typically involves a near field magnetic inductioncommunication system that provides a short range wireless physical layerthat communicates by coupling a tight, low-power, non-propagatingmagnetic field between devices. In such embodiments, the wirelesscommunication module 204 may include a transmitter coil in the in-earutility device 201 to modulate a magnetic field which is measured bymeans of a receiver coil in another device, e.g., another in-ear utilitydevice or a smartphone. In some embodiments, the wireless communicationsmodule 204 can have an ultrasound transducer function, receivingultrasound data communications and translating them into an electronicsignal. Ultrasound communications may offer lower power than some othermodes of wireless communications. The wireless communications module 204may also be capable of translating an electronic signal into anultrasound signal for transmission to another device, according to anembodiment of the invention.

In some embodiments of the invention, the in-ear utility device 201 cancommunicate bi-directionally via a network. In such embodiments, thewireless communications module 204 may comprise a Bluetooth® digitalwireless protocol such that the in-ear utility device 201 maycommunicate with a remote computing device. Bluetooth® technologyprovides a low-cost communication link. The Bluetooth® transceiver in anembodiment of the wireless communications module 204 may be configuredto establish a wireless data link with a suitably equipped mobilecomputing device and/or another in-ear utility device.

In an embodiment, the wireless communications module 204 of the in-earutility device 201 may operate in conjunction with another in-earutility device (e.g. one in-ear utility device in a left ear and anotherin-ear utility device in a right ear), while in another embodiment anin-ear utility device 201 may operate independently. In yet anotherembodiment, at least one in-ear utility device 201 may operate inconjunction with a mobile computing device.

As shown further in FIG. 10, the in-ear utility device 201 may operateas a walkie-talkie device communicating with another in-ear utilitydevice operating in another ear of the user, with another deviceassociated with the user, with another in-ear utility device associatedwith another user, and/or with a third-party device. In someembodiments, a user of the in-ear utility device 201 might be able tocommunicate with another in-ear utility device user using little morethan just a whisper and at great distances.

The in-ear utility device 201 may also include functionality (e.g., thewireless communication module 204) to communicate bi-directionally via along-range wireless network. In one embodiment, the long-range wirelessnetwork includes a cellular network. In another embodiment, thelong-range wireless network includes a multimedia communicationsnetwork. In another embodiment, the long-range wireless network includeswireless technologies such as Global System for Mobile Communications(GSM), Code Division Multiple Access-One (cdmaOne), Time DivisionMultiple Access (TDMA), PDC, Japan Digital Cellular (JDC), UniversalMobile Telecommunications System (UMTS), Code Division MultipleAccess-2000 (cdma2000), and Digital Enhanced Cordless Telephony (DECT).

The wireless communications module 204 may be configured to communicatewith a remote server or network. In one embodiment, the remote networkis a cloud computing platform. As used herein, the term “remotecomputing device” or “mobile computing device” refers to anyone or allof cellular telephones, tablet computers, phablet computers, personaldata assistants (PDAs), palm-top computers, notebook computers, laptopcomputers, personal computers, wireless electronic mail receivers andcellular telephone receivers (e.g., the Blackberry® and Treo® devices),multimedia Internet enabled cellular telephones (e.g., BlackberryStorm®), multimedia enabled smart phones (e.g., Android® and AppleiPhone®), and similar electronic devices that include a programmableprocessor, memory, a communication transceiver, and a display.

Sensors and Sensor Arrays

In embodiments, the in-ear utility device 201 may include one or moresensors 206 a-206 z configured to detect and/or measure variousphenomena. In one embodiment, the in-ear utility device 201 may includeone or more sensors 206 a-206 z configured to detect a physiologicalparameter of the user. Physiological parameters detected or measured bythe sensors 206 a-206 z may include body temperature, pulse, heart rate,VO₂ Max (also known as maximal oxygen consumption), pulse oximetry data,respiratory rate, respiratory volume, maximum oxygen consumption,cardiac efficiency, heart rate variability, metabolic rate, bloodpressure, EEG data, galvanic skin response data, and/or EKG/ECG. Thus,the sensors 206 a-206 z may detect, for example, the ambienttemperature, humidity, motion, GPS/location, pressure, altitude andblood analytes such as glucose of the user of the in-ear utility device201.

In an embodiment, the in-ear utility device 201 may include one or moresensors 206 a-206 z configured to detect the location or motion of theuser, such as, for example an accelerometer, a GPS sensor, a gyroscope,a magnetometer, and/or radiometer. In an embodiment, the in-ear utilitydevice 201 may include a voice sensor 206 a coupled to the microphone203.

Specific sensor 206 a-206 z configurations may vary across embodimentsof the in-ear utility device 201, e.g., one embodiment might include anambient temperature sensor, a heart rate sensor, and a motion sensorwhile another embodiment includes a pressure sensor, a pulse sensor, anda GPS locator. The pulse sensor could be configured to reside on thein-ear utility device 201 in a position inside the user's ear canal andprovide its readings to the processor 207. The pulse measurements couldserve a variety of functions, including as input to a program fordetermining driver alertness, as discussed below.

The various sensor configurations may be configured to work together tomeasure various phenomena that may be used to trigger a particularaction and/or the sensors may operate independently to trigger a varietyof actions. Operations of the sensors 206 a-206 z may be aided by theprocessor 207 and processing instructions 211 stored in the data storagecomponent 209 and/or instructions retrieved from a remote data sourcevia the wireless communications module 204.

The in-ear utility device 201 may be configured to have an array ofsensors 206 a-206 z configured to collect data. The data collected bythe sensors 206 a-206 z may be data related to the user (e.g.,biological data) and/or the user's environment (e.g., temperature,elevation, ambient noise). Such embodiments of the invention do notnecessarily need to include sound processing devices (e.g., amicrophone, a speaker, and/or an amplifier). The collected data may beprocessed by the processor 207 to provide services of use to the userand/or may be transmitted to a device external to the in-ear utilitydevice 201.

Sensors for the in-ear utility device 201 may be provided in packagesthat include extra capacitors and extra insulators for a variety ofhealth and safety reasons, according to an embodiment of invention. Thesensors for the in-ear utility device 201 are typically intended forlong-term in-the-ear usage, according to an embodiment of the invention.For example, the sensor packages in the in-ear utility device 201 may beconfigured such that if a given sensor shorts, the external temperatureof the in-ear utility device 201 will not rise appreciably and the userwill not receive a short, according to an embodiment of the invention.

Equipment Operator Alertness Sensor Array

The sensor(s) 206 a-206 z might include an accelerometer 206 aconfigured to determine if an equipment operator (e.g., a car driver, atruck driver, a locomotive engineer, an airplane pilot, and/or a seacaptain) or someone having a critical need for alertness (e.g., asecurity guard, a policeman, a surgeon, etc.) is sufficiently orappropriately alert to perform their assigned task.

For example, assume that a driver wearing the in-ear utility device 201has fallen asleep while driving and/or is inappropriately alert (e.g.,the driver's head is bobbing up and down), the accelerometer can send anappropriate signal to the processor 207 that executes a safety program211. The accelerometer 206 a may, for example, be configured to measuremovement of the user's head position (see, e.g., FIG. 2C), and theprocessor 207 can use these measurements to determine the operator'salertness. Based on the signal from the accelerometer 206 a andinstructions/data in the safety program 211 (e.g., an alertnessapplication), the processor 207 may determine to take an appropriateremedial action (e.g., sound a warning through the speaker 208). Anin-ear utility device 201 could be configured solely for this function(e.g., no music playing capabilities). Such an embodiment of theinvention does not necessarily require an amplifier and/or a microphone.Alternatively, the in-ear utility device 201 could be configured toperform a variety of functions.

The safety program 211 could be designed in a number of ways and manyexisting operator safety programs already exist. The safety program 211could, for example, receive sensor inputs in addition to theaccelerometer input and integrate the results to form an accurateassessment of the operator's overall alertness state. The safety program211 could also measure the frequency with which the accelerometer 206 areports the driver's head bobbing up and down. If the driver's head bobsup and down X times per minute for some number of minutes, the drivercould be warned and/or an alarm could be sounded, and/or a third partycould be alerted, and/or the vehicle could park itself, assuming thevehicle had capabilities for self-parking or some other stand-down mode.

The processor 207 might also use the wireless communication module 204to send relevant data to another device about the alertness state of thedriver wearing the in-ear utility device 201. So, for example, thewireless communication module 204 might send driver alertness data(and/or alarm states) to a wireless computing device associated with thevehicle controlled by the wearer of the in-ear utility device 201.Depending on the capabilities of the vehicle, the vehicle might opt toengage its own safety program, such as taking control of the vehicleaway from the driver and then parking the vehicle by the side of theroad. The processor 207 may send the driver data to a remote location(e.g., a trucking company's offices and/or an insurance companiesoffices) for remote data processing (e.g., tracking which drivers from afleet of drivers have the greatest tendencies for nodding off whiledriving).

The sensors 206 a-206 z may also include sensors that can assist theprocessor 207 determine who is wearing the in-ear utility device 201.So, for example, the data repository 209 might include biometric data211 related to the user of the in-ear utility device 201. If theprocessor 207 has determined that the user of the in-ear utility device201 is operating heavy equipment, for example, then the sensors 206a-206 z, including the accelerometer, may be configured to track variousactions and alertness states of the user. The in-ear utility device 201may use the communication module 204 to communicate driver informationwith another device (e.g., the vehicle and/or a nearby smartphone) andthen prevent the user from hearing certain things while operating thevehicle. So, for example the vehicle's own safety program might preventthe user from hearing Facebook posts while operating the vehicle. Thesafety program might be configured to prevent everything but emergencymessages from reaching the user of the in-ear utility device 201 whilethe vehicle is in operation and controlled by the user.

The accelerometer may also be used to determine if the user of thein-ear utility device 201 has suffered a strong shock and/or fallendown. When the processor 207 receives a signal from the accelerometerthat matches a pattern for a shock (e.g., fall pattern data 211 storedin the data storage component 209), then the processor 207 may take anappropriate action. So, for example, the processor 207 might instructthe playing of a safety check audio file 211 stored in the data storagecomponent 209 through the microphone 208. The processor 207 might alsoinstruct that the fall condition be transmitted to a distant device(e.g., one operated by a security or health-monitoring company) via thecommunications module 204.

Sensor for Turning in-Ear Utility Device on/Off

The in-ear utility device 201 may include a Hall Effect sensor 219 thatis configured to determine if the in-ear utility device 201 has beeninserted into a charging case (e.g., the charging case 1200 shown inFIG. 12A), and if so, then the processor 207 turns off the in-earutility device 201 (e.g., the Hall Effect sensor 219 acts as a switch),according to an embodiment of the invention.

The Hall Effect sensor 219 includes a transducer that varies its outputvoltage in response to a magnetic field. The Hall Effect sensor 219 canprovide proximity switching, positioning, current sensing applicationsfor the in-ear utility device 201 in connection with a charging case orcharging station for the in-ear utility device 201. The Hall Effectsensor 219 can be used to provide a means for recharging the in-earutility device 201 without the in-ear utility device 201 necessarilyneeding to have a physical on/off switch, and by shutting down thein-ear utility device 201 provides a means for faster recharging of thebattery 213 by the charging case or charging station, according to anembodiment of the invention.

The Hall Effect sensor 219 may be configured to detect the magneticfield of the charging case or charging station as the in-ear utilitydevice 201 is being placed into the recharging device. So, for example,the Hall Effect sensor 219 may detect the charging case at a distance ofone-half inch (12.7 mm.), and the processor 207 may direct electroniccomponents of the in-ear utility device 201 to shut down, according toan embodiment of the invention. An embodiment of a charging case isshown in FIGS. 12A-12D.

The Hall Effect sensor 219 also allows the in-ear utility device 201 tohave a smaller and more stream-lined form factor than might otherwise bethe case—in addition to providing a more efficient means for rechargingthe battery 213. Hall Effect sensors are not conventionally found inBluetooth devices or in hearing aids. The process of using the HallEffect sensor 219 to turn off the in-ear utility device 201 may bereversed when the in-ear utility device 201 is removed from the chargingcase, according to an embodiment of the invention.

Alarm, Notification, and Verification Functions

In another embodiment, the in-ear utility device 201 may provide variousalarm and notification functions. For example, the in-ear utility device201 may be utilized as an alarm clock. This functionality could beprovided by the processor 207 and/or the processor 207 coupled with thedata storage device 209 and/or the processor 207 coupled with thecommunications module 204 and a third device (e.g., a mobile phone). Anordinary artisan should know how to make the processor 207 provide analarm function. In addition, the processor 207 in conjunction, forexample, with data 211 stored in the data storage component 209 mayprovide a calendar function, a timer function, a stopwatch function,and/or a reminder function. Similarly, the processor 207 in combinationwith data 211 from the data storage component 209 combined with datafrom various sensors 206 a-206 z may provide various alarm and/orwarning functions, e.g., a heart attack warning or a high blood pressurewarning. Likewise, in conjunction with the communications module 204,the sensors 206 a-206 z could provide various alarms to various thirdparties remote from the in-ear utility device 201. For example, if thein-ear utility device 201 was equipped with one or more accelerometers206 a, then a third party could be automatically notified of an eventsuch as a car crash, a bicycle crash, and/or a fall.

The in-ear utility device 201 can also be configured to provide variousforms of authentication. Authentication may be provided in a number ofways, including but not limited to application of voice recognitionprocesses known in the art. For example, the microphone 203 incombination with the DSP 212, the processor 207, and the data storagecomponent 209 using voice data 211 can provide authentication of theauthorized user(s) of the in-ear utility device 201. The user couldprovide a voice sample detected by the microphone 203 that is providedto the processor 207 that then retrieves the voice data 211 and comparesthe voice sample against the voice data 211. The processor 207 (possiblyin conjunction with the DSP 212) analyzes the received voice sample anddetermines if the current user's voice matches the voice data 211. Thiselectrical component combination could be used to determine when thein-ear utility device 201 has been stolen or otherwise being operated byan unauthorized person. As mentioned above, the processor 207 could be asimple control circuit configured for the authentication function ratherthan a processor chip configured to control the authentication function.The authentication function could also be used to verify the user beforedelivering sensitive information through the speaker 208.

In an alternative embodiment, authentication may be performed outsidethe in-ear utility device 201 via an external device such as asmartphone. In such an embodiment, the authentication function for thein-ear utility device 201 comprises the microphone 203 and thecommunications module 204 to perform the authentication function in amanner similar to the process described above for an organicauthentication function.

User Interface for in-Ear Utility Device

Sensors, and combinations of sensors 206 a-206 z, may also be used toprovide a user interface function for the in-ear utility device 201. Forexample, an accelerometer 206 a (or a G-force sensor) might activatewhen a user moves or taps his/her hand (or by the user shaking his/herhead while wearing an ear-borne accelerometer in the in-ear utilitydevice) in a predetermined manner (e.g., taps of a certain force orintensity within a certain time frame or head nods of certaincharacteristics) that can be sensed by the accelerometer sensor 206 a.Such an action could trigger the accelerometer sensor 206 a such thatadditional commands might be received through additional actions such asfurther tapping or by head shaking.

For example, a user might tap his/her jaw, ear, check, neck, or anotherpre-designated location (e.g., via a predesignated single tap, doubletap, or triple tap). This tapping action could trigger the accelerometersensor 206 a such that additional commands could also be receivedthrough additional taps. So, for example, once the G-force sensor 206 ahas been activated, then two more taps might activate a music player(e.g., the music player described in FIG. 8) or cause a music selectionto be forwarded by some number of seconds. The taps detected by theaccelerometer 206 a could be delivered to the processor 207 that mayretrieve additional data 211 from the data storage component 209. Theuser's selection could be confirmed by appropriate auditory confirmation(e.g., confirmatory audio message) delivered through the speaker 208.The processor 207 could retrieve an appropriate confirmatory audiomessage 211 from the data storage component 209 and deliver it to thespeaker 208. Choices made by the user as well as possible commandselections could be confirmed (e.g., spoken) to the user via the speaker208 through the use of one or more confirmatory audio messages. Similarsensor configurations 206 a-206 z could also be used for user inputfunctions, such as accelerometers, pulse rate, and temperature sensors.

FIG. 2B provides a diagram 220 that shows a shock (of a certainintensity) provided by a first tap 221 and the shock (of a similarintensity) provided by a second tap 223 as measured by an accelerometersensor 206 a in an in-ear sound device 201, according to an embodimentof the invention. As shown in FIG. 2B, the taps 221, 223 have a tapintensity and a time duration within the predetermined range for a tapcommand recognizable by the in-ear utility device 201 and also include apredetermined quiet period between the taps 221, 223, according to anembodiment. The tap time duration and the quiet period represent apredetermined command convention established by the in-ear utilitydevice 201 for recognizing taps as commands and not ignoring them asbeing merely random shocks. (Of course, the accelerometer 206 a mightrecord all shocks and report them to the processor 207 for anotherpurpose.)

The accelerometer sensor 206 a passes its data to the processor 207shown in FIG. 2A, and the processor 207 compares the received dataagainst relevant command data 211 (e.g., a predetermined pattern) storedin the data storage component 209. If the taps 221, 223 match anappropriate predetermined pattern (e.g., a pattern for predeterminedaction command or predetermined on/off command), then the processor 207engages an appropriate action (e.g., sends an action signal), such asturning on/off the in-ear utility device 201 and/or performing anothertask (e.g., a predetermined action command). For example, arepresentative tap sequence could perform an audio profile selectioncommand that causes the processor 207 to select a given audio profilefrom the data storage component 209, such as the audio profilesdiscussed herein. In some embodiments of the invention, the processor207 may access a confirmatory audio message 211 stored in the datastorage component 209 and play the confirmatory audio message throughthe speaker 208 before engaging any action as a means for determiningthat the user's tap and/or head nod command has been properlyinterpreted by the processor 207.

The accelerometer sensor 206 a might communicate tap data to theprocessor 207 using inter-integrated circuit (I2C) communications,according to an embodiment of the invention. I2C is typically amulti-master, multi-slave, single-ended, serial computer bus that istypically used for attaching lower-speed peripheral integrated circuits(e.g., the accelerometer sensor 206 a) to processors andmicrocontrollers, such as the processor 207. Such communications usebinary code with a unique address through one programmed input/output(PIO). PIO is a method of transferring data between a CPU (e.g., theprocessor 207) and a peripheral (e.g., the accelerometer 206 a). Otherelectric components and sensors 206 a-206 z of the in-ear utility device201 may also use I2C for internal communications, according to anembodiment of invention.

FIG. 2C provides a geometrical representation of tap sensing in whichthe taps are based on distance/tap events/time frame/coordinate,according to an embodiment of the invention. Tap sensing has functionalsimilarity with a common loop touch-pad or clicking keys of a computermouse or tapping on a surface at a specific location, according to anembodiment of the invention. A tap event is detected if a pre-definedslope of the acceleration of the least one axis is exceeded. Twodifferent tap events can be distinguished: A “single tap” is a singleevent within a certain time (e.g., the tap 221 shown in FIG. 2B),followed by a certain quiet time. A “double tap” consists of a firstsuch event (e.g., the tap 221) followed by a second event within adefined time frame (e.g., the tap 223 shown in FIG. 2B).

The orientation recognition feature of the accelerometer 206 a providesinformation about an orientation change of the accelerometer 206 a withrespect to the gravitational field vector ‘g’, as shown in FIG. 2C. Themeasured acceleration vector components with respect to thegravitational field are defined as shown in FIG. 2C. Therefore, themagnitudes of the acceleration vectors may be calculated as follows:

acc_x=1 g×sin θ×cos φ

acc_y=−1 g×sin θ×sin φ

acc_z=1 g×cos θ

acc_y/acc_x=−tan φ

Depending on the magnitudes of the acceleration vectors the orientationof the in-ear utility device 201 in space is determined and stored in anorientation vector. For example, there may be three orientationcalculation modes with different thresholds for switching betweendifferent orientations: symmetrical, high-asymmetrical, andlow-asymmetrical. Additional operational characteristics for theaccelerometer 206 a may be found in manuals, such as BMA222E, Digital,Triaxial Acceleration Sensor for the Bosch Sensortec manufactured byBosch, which is incorporated by reference herein.

In some embodiments of the invention, in the tap sensor user interface,the accelerometer sensor 206 a sends the processor 207 a communicationsequence at periodic intervals that contain received tap data. In otherembodiments of the invention, the tap sensor user interface may bedriven by tap events, e.g., the accelerometer communicates nothing untila tap occurs.

A user interface for the electronic component package 202 shown in FIG.2A, including the sensors 206 a-206 z, could also be provided to theuser via the wireless communications module 204 and an external device,such as a mobile phone or a computer, according to an embodiment of theinvention. A voice command user interface could also be provided via themicrophone 203 and the processor 207, according to an embodiment of theinvention. A voice command user interface could also be provided via thevoice recognition chip 214 applied in combination with the microphone203 with additional data 211 from the data storage component 209 and theprocessor 207, as well as hybrid user interfaces that combine the tapuser interface discussed above with a user interface hosted on a visualdevice, such as a smartphone. An ordinary artisan should understand howto configure these various user interfaces.

The user interface could be provided on a remote device (e.g., asmartphone) with a subset of commands provided by an audio interface inthe in-ear utility device 201. So, for example, commands such as “fastforward” in a music playing apparatus could be engaged through the tapuser interface with more complicated tasks, such as music genreselection, coming from a graphical user interface on a remote device(e.g., a smartphone).

On/Off Sensor Array for in-Ear Utility Device

FIG. 2D illustrates a first humidity sensor 261 and a second humiditysensor 266 that provides a means for turning on/off an in-ear utilitydevice 263 and/or turning on/off components of the in-ear utility deviceusing relative changes in humidity between the two humidity sensors 261,266, according to an embodiment of the invention. The two sensors 261,266 (e.g., the sensors 206 a-206 b shown in FIG. 2A) operate inconjunction with the processor 207 shown in FIG. 2A. The in-ear utilitydevice 263 may be identical to the in-ear utility device 201, accordingto an embodiment of the invention.

The ear canal 264 typically has a different humidity than the ambienthumidity outside the ear 267. The ear canal 264 is typically more humidand warmer than the ambient environment outside the ear, but in anyevent, the two humidities typically differ. Thus, when the humidity in auser's ear canal 264 differs from the humidity outside the ear 267, thenthe processor 207 can conclude that the user is wearing the in-earutility device 263 (e.g., that the in-ear utility device 263 is donnedor worn). Similarly, when the humidity in a user's ear canal 264 matchesthe humidity outside the ear 267, then the processor 207 can concludethat the user is not wearing the in-ear utility device 263 (e.g., thatthe in-ear utility device 263 is not worn or doffed). So, for example,assume that the data threshold trigger for the humidity sensors 261, 266occurs within the range of a 30% congruence in humidity as measured bythe sensors 261, 266, according to an embodiment of the invention.

When the processor 207 receives an indication (e.g., based on receipt ofthe humidity data from the humidity sensors 261, 266) that the humidityreadings have changed to levels indicating that the in-ear utilitydevice 261 has been removed from the ear, then the processor 207 engagesan appropriate action; e.g. the in-ear utility device 261 shuts down, orparticular components of the in-ear utility device shut down, and/or thein-ear utility device 261 switches to a lower energy state, and/or thein-ear utility device 261 attempts to confirm with the user (e.g., viathe speaker 208) that the in-ear utility device 261 is no longer in theear, according to an embodiment of the invention. When the in-earutility device 263 is turned off, then the battery 213 could stillprovide minimal power to the humidity sensor 266 and the humidity sensor261 and the processor 207. If the humidity sensor 266 and/or thehumidity sensor 261 detect changes in humidity indicative of the in-earutility device 263 being put in the user's ear canal 264, then theprocessor 207 can take an appropriate action, e.g., directing thebattery to provide power to more components of the in-ear utility device263.

FIG. 2E illustrates a sensor array 231 that provides an alternativemeans for turning an in-ear utility device on/off using relative changesin temperature and humidity, according to an embodiment of theinvention. The sensor array 231 comprises two sensors (e.g., the sensors206 a-206 b shown in FIG. 2A) that operate in conjunction with theprocessor 207 shown in FIG. 2A.

The sensor array 231 comprises a temperature sensor 235 and a humiditysensor 236. The humidity sensor 236 includes a sensor window 237 thatsamples environmental humidity (e.g., humidity in the ear canal 115shown in FIG. 1A). The temperature sensor 235 and the humidity sensor236 employ I2C data communications (discussed above) to communicatetemperature and humidity data to the in-ear utility device processor(e.g., the processor 207), according to an embodiment of the invention.The sensors 235, 236 may also use PIO to communicate data to theprocessor.

The human ear is generally more humid and warmer than the typicalambient environment outside the ear. Thus, a positive change intemperature and humidity on a relative scale provides an indication thatthe in-ear utility device 261 has gone from not being worn to be worn bya user. Similarly, the reverse indicates that the user has removed thein-ear utility device 261. So, for example, assume that the datathreshold triggers for the temperature chip 235 and the humidity sensor236 are within the range of a 30% to 70% change in humidity ortemperature, according to an embodiment of the invention. As discussedand shown in FIG. 2D, a similar on/off function can be constructed usingtwo humidity sensors at different locations rather than a pairedtemperature and humidity sensor, according to an embodiment of theinvention.

The temperature chip 235 can be configured to monitor the user's bodytemperature. The temperature chip 235 resides inside the user's earcanal (e.g., the ear canal 115 shown in FIG. 1). A typical human's earcanal may include a variety of temperatures, depending on how close tothe tympanic membrane (e.g., the tympanic membrane 104 shown in FIG.1A), with temperatures near the tympanic membrane typically warmer thantemperatures at the outer end of the ear canal. Near the tympanicmembrane the temperature may be 99 degrees Fahrenheit (37 degreesCelsius) while in the middle of the ear canal temperatures may be rangefrom 95-98 degrees Fahrenheit (34-36 degrees Celsius) while at the outerend of the canal may range in temperatures from 60 to 90 degreesFahrenheit (16 to 32 Celsius).

When the processor 207 receives an indication that the temperature haschanged to levels indicating that the user of the in-ear utility device261 has a temperature beyond normal measures (e.g. a fever), then theprocessor 207 engages an appropriate action; e.g. the in-ear utilitydevice 261 sends an audio message 211 (retrieved from the data storagecomponent 209) through the speaker 208 to the user, according to anembodiment of the invention. The in-ear utility device 261 could alsosend temperature data to an external device via the communicationsmodule 204, according to an embodiment of the invention. Thus, thetemperature chip 235 essentially provides a fixed thermometer in theuser's ear canal and could essentially serve as a first warning to theuser of an ill health condition (e.g., becoming too hot while exercisingand/or having a fever due to a cold).

FIG. 2F illustrates how the output from a relative humidity sensor 241and the output from a temperature sensor 243 can be provided to aprocessor, such as the processor 207 shown in FIG. 2A, according to anembodiment of the invention. Assume, for example, that measurements fromthe relative humidity sensor 241 and the temperature sensor 243 undergorespective amounts of signal conditioning 246, 247 before being combinedinto a single data stream.

Assume further that the relative humidity sensor 241 and the temperaturesensor 243 provide their outputs in analog form. Thus, the combinedoutput from the sensors 241, 243 passes through an analog-to-digitalconvertor 245 prior to being provided to a data processing and systemcontrol unit 253 (e.g., the processor 207 shown in FIG. 2A), accordingto an embodiment of the invention. The processor may then provide thedata to various subcomponents and/or take an appropriate action based onthe data received. As discussed above, on/off function could beconfigured using humidity sensors placed at different location on thein-ear utility device. In such an embodiment, the inputs shown in FIG.2F for the temperature sensor might likely not be present because thetemperature sensor would not be needed.

FIG. 3 illustrates an in-ear utility device 303 having a flexible seal302 that covers a portion of the in-ear utility device 303 that isinserted into a user's ear canal (e.g., the ear canal 115 shown in FIG.1A and FIG. 1B) during normal use, according to an embodiment of theinvention.

Embodiments of the invention provide the in-ear utility device 303covered, or partially covered, with the seal 302 that is comfortable towear for a long period of time. The external seal 302 deforms when thein-ear utility device 303 is inserted into a user's ear canal (e.g., theear canal 115 shown in FIG. 1A) without damaging the in-ear utilitydevice 303 or causing harm to the user's ear (e.g., the ear 105 shown inFIG. 1A). The deformable seal 302 cushions the user's ear canal (e.g.,the ear canal 115 shown in FIG. 1A) from the material of the in-earutility device's body 318.

The seals 302 can be produced in bulk eliminating the need forcustomizing the body 318 of the in-ear utility device 303. The seal 302allows the in-ear utility device 303 and its body 318 to be “one sizefits all” and conform to a broad range of ear canal anatomies, accordingto an embodiment of the invention. The seal 302 may be produced inseveral sizes (e.g., small, medium, larger) to accommodate differencesin the size of human ear canals (e.g., the ear canal 115 shown in FIG.1A). The seal 302 may even be produced in more customized sizes (e.g.,from a mold or a measurement) due to the variability in human ear canalsizes.

The seal 302 needs to be comfortable for the user in order for the userto be able to wear the in-ear utility device 303 for long periods oftime. Comfort from the seal 302 comes from making the seal 302 in a sizethat fits well into the user's ear canal. Comfort for the seal 302 alsocomes from making the seals from a flexible and soft material.

The seal 302 can be fabricated from many resilient polymeric materialsknown in the art, according to an embodiment of the invention. There aremany known resilient polymeric materials that may be used to form thein-ear utility device 303 and/or its components, such as the seal 302.For example, natural rubber, neoprene rubber, SBR rubber (styrene blockcopolymer compounds), silicone rubber, EPDM rubber, polybutadienerubber, polyvinylchloride elastomers, polyurethane elastomers, ethylenevinyls, acetate elastomers, elastomers based on acrylic acid precursorsand vinyhalide polymers may all be generally suitable materials whichcan be used to provide the necessary softness for the seal 302. The seal302 covering the in-ear utility device 303 is formed of a material thathas a Shore A Durometer hardness value (by a technique such as ASTM2240-81) of between 20-30, according to an embodiment of the invention.

The seal 302 allows the portion of the body 318 that rests in the user'sear canal (e.g., the ear canal 115 shown in FIG. 1A and FIG. 1B) to benarrower than the ear canal. Thus, the body 318 that contains theelectronic component package 313 does not typically touch the user's earcanal. The presence of the seal 302 protects the user againstmalfunctions of the electronic component package 313. So, for example,if the battery (e.g., the battery 213 shown in FIG. 2A) happens todevelop a short, the user should be protected from shock and heatbecause of the presence of the seal 302. The user is protected by theseal 302 in part because many embodiments of the seal 302 areconstructed from a non-metallic material (i.e., lower heat transfer andpossibly insulating).

The electronic component package 313 may include a speaker 309 disposedat the proximal tip 308 of the in-ear utility device 303. The electroniccomponent package 313 is embedded in the body 318 of the in-ear utilitydevice 303 and includes electronic circuitry configured to perform avariety of tasks and user-engaged functions, including the tasksdescribed in FIGS. 1-2, 4-11.

The in-ear utility device 303 may also include one or more microphoneports (e.g., the microphone port 512 shown in FIG. 5A) to facilitatereceipt of sounds into the in-ear utility device 303, according to anembodiment of the invention. The microphone port 512 is furtherdisclosed in conjunction with FIGS. 5A-5C.

Embodiments of the in-ear utility device 303 have no wires protrudingfrom the body 318 and no external behind-the-ear components. The in-earutility device 303 may be used by the hearing impaired population aswell as the general public. Thus, the in-ear utility device 303 may beused for sound amplification and communication purposes as well as anumber of additional purposes as discussed herein.

The in-ear utility device 303 may also include a joint 312 that swivelsto facilitate insertion of the in-ear utility device 303 in the user'sear, according to an embodiment of the invention. The joint 312 isdescribed further in FIGS. 6A-6C.

FIG. 4 illustrates an in-ear utility device 401 with its deformable sealremoved, according to an embodiment of the invention. As discussed inFIG. 3, the in-ear utility device 302 may include a deformable seal 302for the portion of the in-ear utility device 303 (e.g., an in-earportion 418 b for the in-ear utility device 401) that enters the user'sear.

The in-ear utility device 401 comprises an electronic components package422 that includes a battery 410, a power booster 411, a communicationsmodule (e.g., transceiver) 412, a DSP chip 413, a first microphone 414,a second microphone 415, a voice recognition chip 416, and a noisecancellation chip 417 that provides noise cancellation for the firstmicrophone 414 and/or the second microphone 415, and a speaker 419,according to an embodiment of the invention.

The first microphone 414 may deliver sound to the speaker 419. The firstmicrophone 414 may be in electronic and/or mechanical communication withthe speaker 419. Sound/vibrations picked up by the first microphone 414may be transmitted to the speaker 419 (directly and/or after variousforms of signal processing have been applied to improve the quality ofthe signal). In some embodiments, the sound/vibrations detected by thefirst microphone 414 may be amplified via an amplifier, such as theamplifier 205 shown in FIG. 2A, and transmitted to the speaker 419. Insome embodiments, the amplifier operates in conjunction with the digitalsignal processing (DSP) 413.

The microphones 414, 415 can be used for amplification for the hearingimpaired. Various embodiments of the in-ear sound device 401 can beconfigured to determine which sound sources the user and/or anapplication 211 run by the processor (e.g., the processor 207 shown inFIG. 2A) wants to amplify. If the sounds to be amplified are all soundsin the in-ear utility device's environment, then it makes sense toamplify the signal from the ambient noise microphone 415. If the in-earutility device 401 is configured to amplify the sound from the person(s)to whom the user of the in-ear sound utility device 401 are talking to,then the in-ear utility device 401 would amplify the signal from thevoice microphone 414 since it will be more focused on picking up soundsfrom the direction that the wearer of the in-ear utility device 401 isfacing. This process works well regardless of whether the in-ear utilitydevice 401 is trying to perform noise cancellation based on the ambientmicrophone 415. The in-ear utility device 401 can have multiple modesfor directional amplification such that the in-ear utility device 401can switch among them depending on the situation. The user of the in-earutility device 401 may have an actuator that allows the user to switchbetween modes. The actuator may be engaged by the tap sensor userinterface discussed herein and/or by a visual user interface on a hostdevice, according to an embodiment of the invention. The actuator maycomprise a user-selectable actuator that could be applied to manyembodiments of the invention.

In some embodiments of the invention, the distance between the speaker419 and microphone ports 420 may be at a distance from 15 mm to 5 cm.The distance may need to be adjusted to avoid feedback, depending on thespecific components that are used. As a general matter, the greater thedistance between the microphone ports 420 and the speaker 419, the lowerlikelihood of feedback between the microphone ports 420 and the speaker419.

The power booster 411 supplies additional electrical potential (e.g.,1.4 volts) in order to boost (or amplify) to a higher voltage (e.g., 3volts) the voltage provided by the battery 410 to supply power tocomponents of the in-ear utility device 401 that require higher voltageto operate properly, according to an embodiment of the invention. Asmentioned, power demands for embodiments of the in-ear utility deviceoperate at higher power than a conventional hearing aid.

Voice Recognition and Ambient Sound

The first microphone 414 may focus on picking up the voice of the usermore strongly than the ambient sound microphone 415 while the secondmicrophone 415 may be focused on detecting ambient sound, according toan embodiment of the invention. One or more voice focused ports forreceiving sounds input to the first microphone 414 may reside in anumber of locations on the in-ear utility device 401, such a voicefocused port 512 shown in FIG. 5A.

The voice recognition chip 416 may be configured to perform operationsto distinguish the user's voice from ambient noise. The voicerecognition chip 416 may receive sound signals from the first microphone414, determine whether the sound signals represent the user's voice,activate the speaker 419 when the sound signals represent meaningfulsound, and filter the sounds delivered to the speaker 419 when the soundsignals do not represent meaningful sound.

The voice recognition chip 416 may receive inputs from the firstmicrophone 414 and/or the second microphone 415, according to anembodiment of the invention. As an alternative, the in-ear utilitydevice 401 may include a processor, such as the processor 207 shown inFIG. 2A that has been configured to execute a program 211 that performsoperations to distinguish meaningful sound from ambient noise.

The voice recognition chip 416 (or similar functionality) may beconfigured to engage a phone call, such as answering an incoming phonecall and/or placing a new call, according to an embodiment of theinvention. The voice recognition chip 416 may also provide a capabilityfor disengaging a phone call as well. Similarly, the accelerometersensor 206 a in conjunction with the tap user interface may be used toprovide a capability for engaging/disengaging telephony functions,according to an embodiment of the invention. Telephony functions canalso be engaged through an application on a remote device, such as asmartphone, according to an embodiment of the invention.

Noise Cancellation

The noise cancellation chip 417 filters unwanted ambient noise prior tofiltering the sounds into the first microphone 414 and/or a secondmicrophone 415. The noise cancellation chip 417 may otherwise operate ina conventional manner.

As shown in FIGS. 5A-5C, an in-ear utility device 501 includesmicrophone input ports 512, 514 that take advantage of reflections fromvarious parts of the human ear (e.g., the pinna and the conchae bowl) toensure that the voice focused microphone (e.g., the first microphone414) has a stronger component of relevant voice sound relative to theambient noise microphone (e.g., the second microphone 415) that itselfwill have a stronger component of ambient noise. This difference betweenthe two signals can be used by existing signal processing techniques(e.g., using the processor 207 and/or the DSP 413) to cancel the ambientnoise and thereby optimize the sound quality of the voice data,according to an embodiment of the invention.

By using multiple microphone types for the first and second microphones414, 415, such as an omnidirectional microphone and a directionalmicrophone, the in-ear utility device 401 can exploit the spatialdiversity of the speech and noise relying on spatial information aboutrelative position of speech of interest and the competing noise.Thereby, subtracting the noise from the noisy speech. Thus, the firstmicrophone 414 may comprise an omnidirectional microphone in someembodiments, and the second microphone 415 may comprise a directionalmicrophone in some embodiments.

For embodiments of the invention in which two in-ear utility devicescoordinate their efforts (see, e.g., FIG. 5C), each in-ear utilitydevice 401 may have in each ear (e.g., the ear 105 shown in FIG. 1A) adirectional microphone (e.g., the first microphone 414) facing towardthe user's mouth (see, e.g., FIGS. 5A-5B) and an omnidirectionalmicrophone (e.g., the second microphone 415) that captures sound fromall directions. In such an embodiment, the in-ear utility device 401 mayeither use the first directional microphone (e.g., the microphone 414)to pick up the user's speech or subtract sound coming from thedirectional microphone from the omnidirectional microphone (e.g., thesecond microphone 415) and that should give better information aboutsounds coming from in front of the user. Even though the secondmicrophone 415 is an omnidirectional microphone (capturing sound from360 degrees), due to the location of the microphone (i.e. at entrance ofear canal), it will still see the pinna effect discussed below. Thismeans that sound coming from behind the user will be attenuated andsound from the user's front will be amplified.

The in-ear utility device 401 may employ conventional noise cancellationtechniques that are unrelated to the number of microphones and are basedon spectral analysis of the signal and determining which parts of thesignal that appears likely to be noise. Conventional noise cancellationprocesses employ the scientific certainty that a sound wave and itsinverse cancel each other out. Conventional software, includingavailable open source software, allows noise cancellations systems totake two copies of an audio clip, invert one of them, generate a thirdclip by mixing the two together and the third clip will be silent. Thisis how conventional active noise cancellation methods operate, bycarefully blending the voice microphone signal with the inverted ambientnoise signal making the appropriate adjustments for amplitude, etc.using various conventional signal processing techniques. These functionsmay be carried out by the processor (e.g., the processor 207 shown inFIG. 2A) if not dedicated hardware.

In performing voice recognition, the in-ear utility device 401 mayretain some ambient sound and not filter all of it out before deliveringa combined sound signal to the speaker 419. In other words, the userwill likely want to retain some ambient sound for the user to hear inorder to be fully aware of his environment.

As described further in FIGS. 5A-5C, many different noise cancellationmethods can be employed in the in-ear utility device 401, according toan embodiment of the invention. Among other things, if the user wears anin-ear utility device 401 in the left ear and another in-ear utilitydevice in the right ear, then the two in-ear utility devices can shareaudio data collected by their respective microphones and work togetherto improve voice recognition and/or improve noise cancellation,according to an embodiment of the invention.

The electronics components package 422 may include other combinations ofelectronic components and additional components (as discussed in FIGS.1-3, 5-11), according to an embodiment of the invention. For example,the in-ear utility device 401 may also include a processor and a memorydevice such as the processor 207 and the data storage device 209, shownin FIG. 2A, and/or including one or more sensors 206 a-206 z, accordingto an embodiment of the invention. Among other things, the processorusing data and instructions from the data storage device may performnoise cancellation and/or various enhancement techniques.

The in-ear utility device 401 may also include a swivel joint (e.g., theswivel joint 603 shown in FIG. 6) that connects an out-of-ear portion418 a and the in-ear portion 418 b allowing the out-of-ear portion 418 ato have a different orientation than the in-ear portion 418 b, accordingto an embodiment of the invention. The ability of the out-of-ear portion418 a to have a different orientation from the in-the-ear portion 418 bmay aid in fitting the in-ear utility device 401 into the ear of a user(e.g., the ear 105 shown in FIG. 1A). The in-ear utility device 401 mayinclude more swivels than just a single joint 603, according to anembodiment of the invention. As shown in FIG. 11, the in-ear utilitydevice may alternatively comprise a single piece, according to anembodiment of the invention.

As shown in FIGS. 5A-5B, one or more voice focused port(s) 512 maychannel detected sounds to a microphone focused on picking up the voiceof the user more strongly than the ambient sound (e.g., the firstmicrophone 414 shown in FIG. 4), according to an embodiment of theinvention. The voice focused port(s) 512 may reside on a side 509 of acap end 511 of the in-ear utility device 501, according to an embodimentof the invention. The side 509 resides on an out-of-ear portion 510 a ofthe in-ear utility device 501 that corresponds to out-of-ear portion 418a shown in FIG. 4. The in-ear utility device 501 is shaped such thatwhen inserted into the user's ear canal 515 the voice focused port 512will rest facing outward or forward in alignment with the user's eyesand mouth, as shown in FIG. 5B.

As previously discussed, in some embodiments the port 512 may be locatedinside the user's ear canal 515. FIG. 5B illustrates an in-ear utilitydevice 501 inserted into an ear canal 515, according to an embodiment ofthe invention.

Voice recognition using the voice focused port 512 takes advantage ofthe microphone input port location being a fixed distance from theuser's voice when the user is speaking. As shown in FIG. 5B, a secondportion 510 b of the in-ear utility device 501 (shown here onlypartially) is inserted into the user's ear canal 515 during normaloperation. The first portion 510 a of the in-ear utility device 501having the voice focused port 512 remains outside the user's ear duringoperation but fixed in position because of the anchoring of the secondportion 510 b in the user's ear canal 515, according to an embodiment ofthe invention.

A fixed distance from the voice focused port 512 to the user's mouth 507is useful because this fixed distance helps in setting the spectral andintensity profile of the user's voice for voice recognition purposes andtherefore easier to pick out the user's voice from the incoming audiosignal. Therefore, the fixed distance can improve the signal-to-noiseratio even in noisy environments.

Changing the distance between the microphone input port and the inputsignal affects the signal-to-noise ratio of the captured sound.Moreover, in a reverberant room, the distance between the speaker andthe microphone could also affect the spectrogram of the recorded sound.Therefore, the fact that the in-ear utility device 501 is alwaysrecording the user's voice from a fixed distance makes the speechrecognition easier and more accurate.

The in-ear utility device 501 shown in FIG. 5A has been configured forinsertion into a user's left ear. This orientation means that the voicefocused port 512 shown in FIG. 5A would face the user's mouth in normaloperation, as shown in FIG. 5B. Thus, the voice focused port 512 wouldbe appropriate for the in-ear utility device 501 b shown in FIG. 5C.

The in-ear utility device 501 includes at least one ambient noise port514, according to an embodiment of the invention. The in-ear utilitydevice 501 may even include multiple ambient noise ports 514 (e.g., morethan 10 such ports), according to an embodiment of the invention. Theambient noise ports 514 may be disposed around the exterior of the capend 511 of the in-ear utility device 501 in a 360 degree pattern fromthe center point 517 of the cap end 511 on the outer surface of thein-ear utility device 501, according to an embodiment of the invention.

Among other things, the ambient noise port(s) 514 can support the voicerecognition process by helping cancel out unwanted frequencies in themanner previously discussed. The ambient noise port(s) 514 may provideinput to a microphone, such as the second microphone 415 shown in FIG.4, according to an embodiment of the invention. The ambient noiseport(s) 514 aid in calibrating the direction of sounds 520 a-520 centering the in-ear utility device 501 via the pinna 513 of the ear 505.The pinna 513, or conchae bowl, provides a horn, of sorts, that aids innaturally focusing the sounds 520 a-520 c. The location of the ambientnoise port(s) 514 has been selected to facilitate its operation byadvantageously exploiting the natural focusing and amplification provideby the pinna 513.

Due to the placement of the microphone ports 512, 514 the signal fromthe user's voice is amplified much more than ambient sound, especiallygiven the anatomy of the human ear to which the in-ear utility devicemakes advantageous use of. The pinna 513 has evolved as a tool forenhancing and amplifying sounds having a pitch that is typical for ahuman voice while leaving most other frequencies untouched. Moreover,sounds which are coming from the front of the user sound louder thansounds coming behind the user due, in part, to the construction of theear. Thus, the in-ear sound device 501 has been developed toadvantageously apply the natural condition of the ear 505 and the pinna513. This gives the in-ear sound device 501 the added benefit that thesound from the user's voice sound much louder than any sounds comingfrom behind the user, among other things.

Embodiments of the in-ear utility device 501 may employ directionalmicrophones. Thus, the microphone 414 shown in FIG. 4 and the secondmicrophone 415 shown in FIG. 4 may be directional microphones. Asdiscussed with regard to the microphone ports 512, 514, one of theseports, the voice focused port 512 is specifically aimed at the user, andthe ambient noise port(s) 514 are aimed straight in the vicinity of thespeaker. Depending on whether the in-ear utility device 501 wants tofocus on the user's voice or the sounds coming to the user and theuser's environment, the signals from each of the microphones 414, 415can be subtracted from each other, and the signal from the microphonethat is of interest can be amplified.

The fact that one of the microphone input ports is in the ear canal 515allows for cues from the pinna 515 which can be applied for front/backlocalization by the processor (or combination of equipment performingthe sound processing functions). Moreover, use of directionalmicrophones may also help in front/back localization of the speaker ofinterest. In addition, using of the right in-ear utility device 505 aand the left in-ear utility device 505 b (discussed in FIG. 5C) improvessound localization of right/left differentiation.

The microphone ports 512, 514 could be placed in a variety of locationson the in-ear utility device 501. The microphone ports 512, 514 couldeven be located inside the portion of the in-ear utility device 501 thatresides in the user's ear canal 515. One microphone port, for example,could face inward to the user's ear canal, which facilitates determiningwhen the user is speaking. The in-ear utility device 501 could eveninclude a bone conduction microphone. In some embodiments of theinvention, the ambient noise port(s) 514 could be replaced with a signalport, such as the embodiment shown in FIGS. 5F-5H.

Many different noise cancellation methods can be employed in the in-earutility device 501, as discussed in conjunction with FIG. 4. Among otherthings, if the user wears an in-ear utility device 501 in the left earand another in-ear utility device in the right ear (as shown in FIG.5C), then the two in-ear utility devices 501 a, 501 b can be configuredto work together to improve speech recognition and/or improve noisecancellation, according to an embodiment of the invention. On someoccasions, the user of the in-ear utility device 501 may simply want touse the in-ear utility device 501 to cancel all the sound around himselfand not hear any speech.

For example, left/right sound localization cues in the horizontal areobtained from interaural level differences and interaural timedifferences between the right ear 501 a and left ear 501 b, which may beadvantageously used by the right in-ear utility device 505 a and theleft in-ear utility device 505 b. Maintaining these cues by applyingsimilar signal processing (e.g., the DSP 212 shown in FIG. 2A) on boththe right in-ear utility device 505 a and left in-ear utility device 505b help with sound localization of the speaker of interest, e.g., theuser of the in-ear utility device. Therefore, results may be improvedwhen the right in-ear utility device 505 a communicates with the leftin-ear utility device 505 b (e.g., the right in-ear utility device 505 atransfers external sounds to the left in-ear utility device 505 b). Withimproved sound localization of the speaker of interest (e.g., the userof the in-ear utility device), speech understanding and communicationmay improve, according to an embodiment of the invention.

When the right in-ear sound device 505 a communicates (e.g., transfersexternal sounds) with the left in-ear sound device, binaural beamformingcan be conducted to narrow the directional focus of the beam so thatanything outside that region is attenuated which improve thesignal-to-noise ratio significantly and improves speech recognition,according to an embodiment of the invention. Embodiments of theinvention that perform beamforming may include two microphones perin-ear utility device 505 a, 505 b, two microphones for the right in-earutility device 505 a and two microphones for the left in-ear utilitydevice 505 b, and the ports for the microphones may typically be locatedat some distance away from each other in the in-ear utility device 505a, 505 b.

Moreover, once the source of the speech of interest is determined (e.g.,the user of the in-ear sound device), the in-ear sound devices 505 a,505 b can amplify the speech of interest and reduce the surroundingsound for further improvements in speech intelligibility.

The in-ear utility device 501 may communicate (e.g., via thecommunication module 204 shown in FIG. 2A) with a counterpart in-earutility device (e.g., an in-ear utility device 501 b in the left earcommunicating with an in-ear utility device in the right ear 501 a) toimprove overall functionality. For example, the microphone(s) in theleft ear in-ear utility device 501 b may combine received sounds withthe microphone(s) in the right ear in-ear utility device 501 a. Inputsfrom these multiple microphones may improve overall noise cancellationfor each in-ear utility device 501 a, 501 b.

Similarly, microphones in either or both of the in-ear utility devices501 a, 501 b may be placed in different locations. Placing themicrophones in different locations allows different sound signals to bereceived by the in-ear utility device 501, and these different signalsmay facilitate noise cancellation.

Using voice profiles (e.g., voice profiles 211 stored in the datastorage component 209 shown in FIG. 2A), a processor in the in-earutility device 501 (e.g., the processor 207 shown in FIG. 2A) can employnoise cancellation to identify a very specific sound in a haze of noise(e.g., picking a particular person out in a crowd). So, for example,assume a user of the in-ear utility device 501 attends a concert withhis/her spouse. Assume further that the in-ear utility device 501 has avoice profile of the spouse. By applying the voice profile for thespouse (e.g., a voice profile 211 stored in the data storage component209), the in-ear utility device's noise cancellation process can use thevoice profile as a filter to cancel sounds not fitting the voice profileand thereby allow the user to hear the spouse's voice at a greaterdistance in a noisy crowd than would be the case without the additionalprocessing or with the unaided ear.

Voice profiles could take a number of different formats but typicallyinclude information regarding the tonality, cadence, and frequencyresponse of the person associated with the voice profile. Creating suchprofiles are not a part of the invention herein; however, such voiceprofiles can be created by having a person record a small number ofsentences and then analyzing the response to derive the essentialcharacteristics of the person's voice. Embodiments of the in-ear utilitydevice 501 could obtain and store a large number of voice profiles(e.g., in the storage device 209 shown in FIG. 2A). Voice profiles areone representative embodiment of an audio profile, which could be asimilar profile for some sound (human, animal, machine, etc.) that isamenable to being used as a filter; thus, the voice profiles discussedherein are representative examples of audio profiles.

The enhancement of a speaker's voice can be performed in a number ofways. For example, from a spectrogram of a speech, the pitch range,intonational pattern, voice quality, dialectal traits of the speaker canbe obtained. In other words, the characteristics of the speaker's voiceor voice biometrics can be gleaned.

If the data storage component of the in-ear utility device (e.g., thedata storage component 209 shown in FIG. 2A) has a database of differentpeople's voice profiles (e.g., based on voice biometrics), then theprocessor (e.g., the processor 207 shown in FIG. 2A) can identify aparticular speaker in a speech sample. Once the speaker of interest isdetermined, then the incoming sound captured by the in-ear utilitydevice 501 can be filtered by the characteristics of the speaker ofinterest and that received sound signal can be amplified under thedirection of the processor and all other sounds can be filtered ordiminished. Using statistical models of speech and noise, once theprocessor of in-ear utility device 501 knows the temporal and spectralcharacteristics of speech of interest, the processor can engage thefiltering out of all other sounds.

FIG. 5C illustrates a portion of the distal ends of two in-ear utilitydevices 501 a, 501 b in a single user's ears 505 a, 505 b in a head 554,according to an embodiment of the invention. FIG. 5C shows the right ear505 shown in FIG. 5B and adds a left ear 505 b.

The distal ends of the in-ear utility devices 501 a, 501 b provide afixed distance from the user's mouth 507 since the in-ear utilitydevices 501 a, 501 b are anchored in the user's ear canals. Aspreviously shown in FIG. 1A and FIG. 1B and FIGS. 5A-5B, the in-earutility devices 501 a, 501 b are placed in the user's ear canal (e.g.,the ear canal 115 shown in FIG. 1A) during operation and are far lesssubject to movement once placed in the user's ears 505 a, 505 b.

Thus, the in-ear utility device 501 essentially resides at a fixeddistance from the user's mouth 507. The fixed proximity to the user'smouth 507 coupled with the stability of the fixed distance simplifiescalibration of the user's voice by the processor (e.g., the processor207 shown in FIG. 2A) and simplifies recognition of the user's voice.

Sounds from the user's mouth 507 can be focused and amplified byallowing the in-ear utility devices 501 a, 501 b to advantageously applythe natural focusing and amplification by the pinna 513 a, 513 b of theuser's ears 505 a, 505 b, as shown in FIG. 5B, especially the conchaebowl portion of the pinna. Here, sounds from the user's voice 520 a-1,520 b-1, 520 c-1 traveling to the user's right ear 505 a can becollected and focused naturally by the pinna 513 a in the user's rightear 505 a before entering a microphone port on the in-ear utility device501 a. Similarly, sounds from the user's voice 520 a-2, 520 b-2, and 520c-2 traveling to the user's left ear 505 b can be collected and focusednaturally by the pinna 513 b in the user's left ear 505 b beforeentering a microphone port on the in-ear utility device 501 b.

FIG. 5D illustrates a top-down view of in-ear sound devices 505 a, 505 bperforming binaural beamforming for sounds in front of the head 554shown in FIG. 5C, according to an embodiment of the invention. When thein-ear sound devices 505 a, 505 b perform binaural beamforming, then thein-ear sound devices 505 a, 505 b will particularly focus on sounds infront of the user's head 554 and will in particular focus on sounds 550essentially pointed to by the user's nose 552.

The in-ear sound device 505 a and the in-ear sound device 505 b may bepaired with each other, according to an embodiment of the invention. Oneof the in-ear sound devices may serve as a master device while the otherdevice serves as a slave device. Microphone inputs from the in-ear sounddevices 505 a, 505 b can be combined (e.g., in the master in-ear sounddevice) so that signal processing (e.g., using DSP 212 shown in FIG. 2A)can be performed on the microphone inputs so as to pick out a specificobject (e.g., a person) that the user wants to concentrate on (e.g., viabeamforming) and/or to improve signal-to-noise ratio in the combinedsignal, according to an embodiment of the invention.

When the right in-ear sound device 505 a communicates its sound inputswith the left in-ear sound device 505 b, binaural beamforming can beconducted to narrow the directional focus of the beam so that anythingoutside a region in an arc around the front of the user's head isattenuated, which improves the signal-to-noise ratio significantly andimproves speech recognition, according to an embodiment of theinvention.

Embodiments of the invention that perform beamforming may include atleast two microphones per in-ear utility device 505 a, 505 b, e.g., twomicrophones for the right in-ear utility device 505 a and twomicrophones for the left in-ear utility device 505 b.

The ports for the microphones may typically be located at some distanceaway from each other in the in-ear utility device 505 a, 505 b. Forexample, the microphone port for ambient sound may be located on theopposite side of the in-ear utility device from the voice focused port,such as the voice focused port 512 shown in FIG. 5A. In other words, insome embodiments of the invention, an outwardly facing ambient noiseport (such as the ambient noise ports 514) might be replaced (orsupplemented) by an ambient noise port at a location opposed to thevoice focused port.

In some embodiments, it may be simpler to have two ambient noise ports(e.g., one outwardly facing and one opposed to the voice focused port)and two ambient noise microphones with a controller (e.g., the processor207 shown in FIG. 2A) that simply switches one ambient microphone offand another on, depending on whether the in-ear utility device isperforming binaural beamforming or a similar function as opposed toperforming a task optimized by an outwardly facing ambient microphoneport. Of course, it would also be possible to use a smaller number ofmicrophones and have some sort of physical device that opened and closedthe various input ports depending upon their function.

FIG. 5E illustrates an in-ear utility device 560 having two ambientnoise microphones 562, 564 and two ambient noise microphone ports 561,563 along with a voice focused microphone 566 and a voice focusedmicrophone port 565, according to an embodiment of the invention. Theambient noise port 561 is on the opposite side of the in-ear utilitydevice 560 from the voice focused microphone port 565. The ambient noiseport 563 can take a number of forms and shapes, including those shown inFIG. 5A and FIG. 5F, and need not be a single port but could be an arrayof ports as well.

The voice-focused microphone port 565 may typically come to rest insidethe user's ear canal 515 at a portion facing the user's mouth 507 and ina region to benefit from natural amplification provided by the pinna 513of the user's ear 505, according to an embodiment of the invention. Theambient noise port 561 may typically come to rest inside the user's earcanal 515 at a portion facing away from the user's mouth 507 and in aregion to benefit from natural amplification provided by the pinna 513of the user's ear 505, according to an embodiment of the invention.Instructions for proper placement of the in-ear sound device 560 may beprovided to the user via various instructional materials. In addition,the accelerometer sensor 206 a may be configured to determine theorientation of the ambient noise port 561 and the voice-focusedmicrophone port 565. For example, the accelerometer sensor 206 a,working with the processor and the speaker (e.g., the processor 207 andthe speaker 208 shown in FIG. 2A), could signal a beeping sound when thein-ear sound device has an acceptable orientation, according to anembodiment of the invention.

A control circuit or processor (such as the processor 207 shown in FIG.2A) controls which microphones 562, 564, 566 are operating at any giventime, e.g., by turning off the power to these microphones. When thepower to a given microphone is turned off, then sounds entering throughthe microphone's respective port will be not be amplified andessentially ignored.

For example, the processor switches on microphones 562, 566 when thein-ear utility device 560 is performing a beamforming function, and themicrophone 564 is similarly switched off for this task. Similarly, theprocessor may switch off microphones 562, 566 and switch on microphone564 if the in-ear utility device is focused on an ambient noiselistening task. Likewise, microphones 562, 564 may be switched off, andmicrophone 566 turned on when the in-ear utility device 560 is focusedon a voice recognition task, according to an embodiment of theinvention. The processor may also switch on microphones 564, 566 forcertain listening tasks, according to an embodiment of the invention. Asdiscussed above, when one of the microphones 562, 564, 566 is switchedoff, then sounds entering a respective port 561, 563, 565 are notprocessed and essentially ignored.

The processor may turn microphones on and off fairly rapidly, allowingthe in-ear utility device 560 to perform a number of functions and tasksnearly concurrently, according to an embodiment of the invention. Inaddition, as discussed herein, these various hearing tasks may becontrolled by the user via various voice commands received by the in-earutility device, according to an embodiment of the invention.

FIGS. 5F-5H illustrate a 360-degree slit port 524 for the microphoneport on a distal end 522 of an in-ear utility device 520, according toan alternative embodiment of embodiment of the invention. The slit port524 provides an opening all around the distal end 522 and directs soundsto a microphone (e.g., the first microphone 414 shown in FIG. 4).

FIG. 5G illustrates a cross-section 526 of the distal end 522 of thein-ear utility device 520 showing the slit port 524, according to anembodiment of the invention. A top portion 532 of the slit port 524 issuspended slightly above a bottom portion 534. The top portion 532 canbe fastened to the bottom portion 534 by a variety of fasteners 536. Thefastener 536 could be a hook, a hinge, a tongue that is glued to connectthe top portion 534 to the bottom portion 536, a piece that is melted,or even formed as an integral piece, etc.

The bottom portion 534 can be attached to the body of the in-ear utilitydevice 520 by a number mechanisms, such as a hook 538 that slides intoanother piece on the in-ear utility device 520, according to anembodiment of the invention.

FIG. 5H illustrates dimensions of the 360-degree slit port 524 on thedistal end 522 of the in-ear utility device 520, according to anembodiment of the invention. A top portion 532 may have a smallerdiameter in comparison to the inner diameter of the bottom portion 534such that the two pieces are separated by a distance 540. The distance540 may range from 1.0 to 2.7 millimeters, according to an embodiment ofthe invention. The top portion 532 may be raised above the bottomportion 534 by a distance 542. The distance 542 may range from 0.1 to0.5 millimeters, according to an embodiment of the invention.

FIGS. 6A-6C illustrate a swivel joint 603 in the in-ear utility device601 that allows the in-ear utility device 601 to pivot from zero(vertical) to negative 30 degrees and from zero to plus 30 degrees. Theswivel joint 603 may pivot to other ranges of degrees in otherembodiments of the invention.

Human ears often have ear canals (e.g., the ear canal 115 shown inFIG. 1) of various shapes and sizes (e.g., an s-shaped ear canal). Thus,the swivel joint 603 facilitates placing the in-ear utility device 601in the user's ear securely.

FIG. 6A shows the in-ear utility device in a −30 degree position. FIG.6B shows the in-ear utility device 601 in a zero position, and FIG. 6Cshows the in-ear utility device 601 in a +30 degree position, accordingto an embodiment of the invention.

The swivel joint 603 allows the in-ear utility device 601 to pivot intoan ear canal (e.g., the ear canal 115 shown in FIG. 1A) that is twistedand not straight, according to an embodiment of the invention.Embodiments of the in-ear utility device could pivot to a greater and/orsmaller degree, e.g., from 10 degrees to 90 degrees.

Some users of the in-ear utility device 601 may have ear canals that arenot straight but spiral upwards or downwards or both. The in-ear utilitydevice 601 having the swivel joint 603 may be useful to users havingstraight ear canals but will be especially useful for user's havingspiral ear canals.

The swivel joint 603 allows the in-ear utility device 601 to move closerto the user's eardrum (e.g., the eardrum 104 shown in FIG. 1A). Thecloser the in-ear utility device 601 resides to the user's eardrum 104,the smaller amounts of power the in-ear utility device 601 needs tooperate the speaker (e.g., the speaker 108), according to an embodimentof the invention. In addition, the closer proximity of the in-earutility device 601 to the user's eardrum may also increase the qualityof sound delivered by the in-ear utility device 601.

The in-ear utility device 601 may have more than one swivel joint 603.Additional swivels placed in the in-ear utility device 601 will increasethe degrees of freedom offered by the in-ear utility device 601. It isknown in the art, that some users have ear canals that are so spiralingthat they double back on themselves. Such users will particularlybenefit from the increased degrees of freedom provided by the swivel(s)in the in-ear utility device, according to an embodiment of theinvention.

FIG. 7 illustrates an embodiment of an in-ear utility device 701configured to function as a headphone, according to an embodiment of theinvention. The in-ear utility device 701 may have the shape and otherergonomic characteristics of the in-ear utility device 303 shown in FIG.3, for example.

The in-ear utility device 701 comprises a speaker 708, a battery 713, acommunication module 704, and a control circuit 707 in an electroniccomponent package 702. The in-ear utility device 701 may compriseadditional electronic components in the headphone embodiment. However,the components provided here are sufficient to enable an in-earheadphone capability. The electronic component package 702 is placed inor on a body 710 designed for comfortably wearing for long periods oftime, according to an embodiment of the invention. The electroniccomponent package 702 does not need to be inserted as a single unit;individual components may be inserted individually, for example.

The control circuit 707 may operate in a conventional manner for suchcircuits, controlling the receipt of data (e.g., music or voice data)from outside the in-ear utility device 701 via the communication module704 and directing transfer of the data to the speaker 708, withoperations powered by the battery 713. The control circuit 707 may insome embodiments comprise a dedicated computer chip (e.g., the processor207 shown in FIG. 2A) configured to provide equivalent or superiorfunctionality to a control circuit, according to an embodiment of theinvention.

FIG. 8 illustrates an embodiment of an in-ear utility device 801configured to function as a music player, according to an embodiment ofthe invention. The in-ear utility device 801 may have the shape andother ergonomic characteristics of the in-ear utility device 303 shownin FIG. 3, for example.

The in-ear utility device 801 comprises a speaker 808, a battery 813, adata storage component 809, and a control circuit 807 in an electroniccomponent package 802. The in-ear utility device 801 may compriseadditional electronic components in the music player embodiment.However, the components provided here are sufficient to provide a musicplayer capability. The data storage component 809 includes music data811. The electronic component package 802 is placed in or on a body 810.The electronic component package 802 does not need to be inserted as asingle unit; individual components may be inserted individually, forexample.

The control circuit 807 may operate in a conventional manner for suchcircuits, obtaining music data 811 from the data storage component 809and directing transfer of the music data 811 to the speaker 808, withoperations powered by the battery 813. The control circuit 807 may insome embodiments comprise a dedicated computer chip (or processor)configured to provide equivalent or superior functionality to a controlcircuit, according to an embodiment of the invention. The in-ear utilitydevice 801 may include a communications module, such as thecommunications module 204 shown in FIG. 2A. The communications modulemay provide a means for storing new music. Alternatively, a port couldallow music to be directly added to the data storage component 809.

FIG. 9 illustrates an embodiment of an in-ear utility device 901configured to provide hearing amplification, according to an embodimentof the invention. The in-ear utility device 901 may have the shape andother ergonomic characteristics of the in-ear utility device 303 shownin FIG. 3 for example.

The in-ear utility device 901 comprises a speaker 908, a battery 913, amicrophone 903, an amplifier 905, and a control circuit 907 in anelectronic component package 902. The in-ear utility device 901 maycomprise additional electronic components in the hearing amplificationembodiment, such as a digital signal processor (DSP) 912. However, thecomponents provided here are sufficient to provide an in-ear hearingamplification capability. The electronic component package 902 is placedin or on a body 910 design for comfortably wearing for long periods oftime, according to an embodiment of the invention. The electroniccomponent package 902 does not need to be inserted as a single unit;individual components may be inserted individually, for example.

The control circuit 907 may operate in a conventional manner for suchcircuits, receiving sound data from the microphone 903, directingtransfer of the data to the amplifier 905 (and possibly the DSP 912),and then directing the amplified and/or enhanced sound to the speaker908, with operations powered by the battery 913. The control circuit 907may in some embodiments comprise a dedicated computer chip (orprocessor) configured to provide equivalent or superior functionality toa control circuit, according to an embodiment of the invention. In someembodiments, the control circuit 907 may also direct the operations ofthe DSP 912.

The in-ear utility device 901 may include additional microphones, asdiscussed above, and the microphones may have specialized portsdepending upon their specific function, as discussed above. Embodimentsof the in-ear utility device 901 may also include a voice recognitionchip along the lines of the voice recognition chip previously discussed.

FIG. 10 illustrates an embodiment of an in-ear utility device 1001configured to provide a walkie-talkie function (a portable, two-wayradio transceiver), according to an embodiment of the invention. Thein-ear utility device 1001 may have the shape and ergonomiccharacteristics of the in-ear utility device 303 shown in FIG. 3, forexample.

The in-ear utility device 1001 comprises a speaker 1008, a battery 1013,a microphone 1003, a communication module 1004, and a control circuit1007 in an electronic component package 1002. The in-ear utility device1001 may comprise additional electronic components in the walkie-talkieembodiment. However, the components provided here are sufficient toprovide an in-ear walkie-talkie capability. The electronic componentpackage 1002 is placed in or on a body 1010 designed for comfortablywearing for long periods of time, according to an embodiment of theinvention. The electronic component package 1002 does not need to beinserted as a single unit; individual components may be insertedindividually, for example.

The control circuit 1007 may operate in a conventional manner for suchcircuits, receiving sound data from the microphone 1003, directingtransfer of the data to the speaker 1008, with operations powered by thebattery 1013. The control circuit 1007 may also send the audio datareceived by the microphone 1003 to a remote device using thecommunications module 1004. The control circuit 1007 may also receiveaudio data from the communication module 1004 and direct the audio datato the speaker 1008.

The control circuit 1007 may in some embodiments comprise a dedicatedcomputer chip (or processor like the processor 207 shown in FIG. 2A)configured to provide equivalent or superior functionality to a controlcircuit, according to an embodiment of the invention.

FIG. 11 illustrates an embodiment of an in-ear utility device 1101configured in a single, integrated body 1118 rather than as amulti-pieced body as shown and described in FIG. 3-5. The integratedbody 1118 of the in-ear utility device 1101 includes a microelectronicscomponent package 1113. The in-ear utility device 1101 is shown in FIG.11 with the body 1118 separated from a seal 1102 that typically coversat least a tip end of the body 1118 when the in-ear utility device 1101is inserted into the user's ear canal (e.g., the ear canal 115 shown inFIG. 1A).

Embodiments of the in-ear utility device 1101 may fit completely insidethe user's ear canal (e.g., the ear canal 115 shown in FIG. 1A) with nopart of the device extending outside the user's ear. The in-ear utilitydevice 1101 may include a ring 1111 that facilitates removal of thedevice from the user's ear, e.g., the in-ear utility device 1101 may beremoved by latching the ring 1111 with a small utility device having amatching hook. In an alternative embodiment, the body 1118 may be madeof a metallic substance such that the in-ear utility device 1101 can beremoved from the user's ear using a magnet.

Embodiments of the invention provide an in-ear utility device 1101covered, or partially covered, with a seal 1102 that is comfortable towear for a long period of time. The seal 1102 can be produced in bulkeliminating the need for customizing the body 1118 of the in-ear utilitydevice 1101. The external seal 1102 deforms when the in-ear utilitydevice 1101 is inserted into a user's ear canal (e.g., the ear canal 115shown in FIG. 1A) without damaging the in-ear utility device 1101 orcausing harm to the user's ear (e.g., the ear 105 shown in FIG. 1A).

The deformable seal 1102 cushions the user's ear canal (e.g., the earcanal 115 shown in FIG. 1A) from the material of the in-ear utilitydevice's body 1118, allowing the user to wear the in-ear utility device1101 for an extended period of time. The seal 1102 allows the body 1118of the in-ear utility device 1101 to be a “one size fits all” andconform to a broad range of ear canal anatomies, according to anembodiment of the invention. The seal 1102 may be produced in severalsizes (e.g., small, medium, larger) to accommodate differences in thesize of human ear canals (e.g., the ear canal 115 shown in FIG. 1A).

The electronic component package 1113 is embedded in the body 1118 ofthe in-ear utility device 1101 and includes electronic circuitryallowing the in-ear utility device 1101 to be inserted into the user'sear canal (e.g., the ear canal 115 shown in FIG. 1A) without damagingthe in-ear utility device 1101 or causing injury to the user's ear,according to an embodiment of the invention.

The electronic component package 1113 may include a speaker 1109disposed at the proximal tip 1108 (e.g., the proximal tip 107 shown inFIG. 1A) of the in-ear utility device 1101. The speaker 1109 is disposedat the proximal tip of the body 1118, and when the seal 1102 is fittedonto the in-ear utility device 1101, the proximal tip 1108 for thein-ear utility device 1101 becomes the seal 1102, according to anembodiment of the invention.

Embodiments of the in-ear utility device 1101 have no wires protrudingfrom the body 1118 and no external behind-the-ear components associatedwith the in-ear utility device 1101. The in-ear utility device 1101 maybe used by the hearing impaired population as well as the generalpublic. Thus, the in-ear utility device 1101 may be used for soundamplification and communication purposes as well as a number ofadditional purposes, such as those previously discussed herein.

The in-ear utility device 1101 may also include a microphone port (e.g.,the microphone port 512 shown in FIG. 5) to facilitate receipt of soundsinto the in-ear utility device 1101, according to an embodiment of theinvention. The in-ear utility device 1101 may have other ports,including ports for specific purposes, such as voice receipt/recognitionand ambient noise receipt.

In-Ear Utility Device Recharging Case

FIGS. 12A-12D illustrate a recharging case 1200 configured to recharge apair of in-ear utility devices 1201, 1202, according to an embodiment ofthe invention.

FIG. 12A illustrates a pair of in-ear utility devices 1201, 1202inserted into charging ports 1210, 1211 in the bottom 1208 of thecharging case 1200. The lid 1204 includes two magnets 1205, 1206,according to an embodiment of the invention. The magnets 1205, 1206 aremounted in the lid 1204 in alignment with the charging ports 1210, 1211.Thus, the magnets 1205, 1206 will be above the in-ear utility devices1201, 1202 when the in-ear utility devices 1201, 1202 are in thecharging ports 1210, 1211, according to an embodiment of the invention.

When a lid 1204 is open, the magnetic field from the magnets 1205, 1206will not engage the Hall Effect sensors in the in-ear utility devices,e.g., the Hall Effect sensor 219 shown in FIG. 2A.

As shown in FIG. 12B, when the lid 1204 is shut, the magnets 1205, 1206trigger the Hall Effect sensors in the in-ear utility devices 1201,1202. As discussed with the Hall Effect sensor 219 shown in FIG. 2A, theHall Effect sensors engage the turning off of the in-ear utility devices1201, 1202.

The in-ear utility devices 1201, 1202 may include LEDs 1212, 1213,according to an embodiment of the invention. When the user opens the lid1204 after the in-ear utility devices 1201, 1202 have re-charged, theHall Effect sensor (e.g., the Hall Effect sensor 219 shown in FIG. 2A)will engage the turning on of the in-ear utility devices 1201, 1202.Turning on the in-ear utility devices 1201, 1202 will also cause theLEDs 1212, 1213 to turn on. The LEDs 1212, 1213 will flash until thein-ear utility device 1201 has been paired with the in-ear utilitydevice 1202, according to an embodiment of the invention. Once thein-ear utility devices 1201, 1202 have been paired with each other, thenthe LEDs 1212, 1213 will stop flashing.

The in-ear utility devices 1201, 1202 may be paired with each otherusing a third device (e.g., a smartphone), according to an embodiment ofthe invention. The in-ear utility devices 1201, 1202 may also be pairedwith each other using a pairing circuit included in the charging case1200, according to an embodiment of the invention.

FIG. 12C provides a side view of the charging case 1200 with the lid1204 closed while the in-ear utility device 1201 charges in the chargingport 1210, according to an embodiment of the invention.

The magnetic field produced by the magnet 1205 should be perpendicularto the Hall Effect sensor (e.g., the Hall Effect sensor 219 shown inFIG. 2A) in the in-ear utility device 1201 in order for the magnet 1205to activate the Hall Effect sensor. Thus, the magnet 1205 and thecharging port 1210 should be positioned inside the charging case 1200such that the Hall Effect sensor in the in-ear utility device 1201 isperpendicular to the magnet 1205, according to an embodiment of theinvention. As discussed above, when the lid 1204 is closed, the magneticfield activates the Hall Effect sensor, causing the in-ear utilitydevice 1201 to turn off, and the LED 1212 (not shown in FIG. 12C willalso not flash.

FIG. 12D illustrates a micro-USB port 1214 that can provide power to thecharging case 1200, according to an embodiment of the invention. Oncethe in-ear utility devices 1201, 1202 are changed, the user can unplugthe charging case 1200 and take the case 1200 with him/her. Depending onthe battery type in the in-ear utility devices 1201, 1202, the in-earutility devices 1201, 1202 will be able to retain their charge for someperiod of time in the case 1200 after being recharged, according to anembodiment of the invention.

FIG. 13 illustrates a network 1300 through which various processingtasks for in-ear utility devices 1301 a, 1301 b can be distributed,according to an embodiment of the invention. Some processing tasks canbe performed by the processors on the in-ear utility devices 1301 a,1301 b; other processing tasks can be performed by a remote device, suchas a smartphone 1314, and other processing tasks can be performed by apowerful remote computing facility 1318 (e.g., a cloud computingnetwork), according to an embodiment of the invention.

A user may wear in-ear utility devices 1301 a, 1301 b in each ear 1305a, 1305 b. In some configurations, one of the in-ear utility devices(e.g., the in-ear utility device 1301 a) may serve as a master devicebetween the two in-ear utility devices 1301 a, 1301 b, according to anembodiment of the invention. In other embodiments, each in-ear utilitydevice may operate independently and communicate independently withremote devices, such as the smartphone 1314, and the remote computingfacility 1318.

The processor (e.g., the processor 207 shown in FIG. 2A) in an in-earutility device (e.g., the in-ear utility device 1301 a) may beprogrammed to have an understanding of tasks that it can complete itselfand tasks that should be completed by a remote device. So, for example,if the user asks the in-ear utility device 1301 a, “Where is the nearestrestaurant?” the processor on the in-ear utility device 1301 a mayrecognize the utterance as an instruction. However, the processor mayalso recognize that this is a command that it cannot process alone.

Consequently, the processor passes the command to either the smartphone1314 and/or the remote computing facility 1318, according to anembodiment of the invention. The remote computing facility 1318 maylocate the requested information and return the answer to the in-earutility device, which then delivers the answer to the speaker of thein-ear utility device. As previously discussed, the in-ear utilitydevice may communicate to the smartphone 1314 using a protocol such asBluetooth and may communicate to the remote computing facility 1318,possibly via a mobile base station 1316, according to an embodiment ofthe invention. The in-ear utility device 1301 a may communicate to themobile base station 1316 using a protocol such as GSM, according to anembodiment of the invention.

Any number of tasks may be performed on the in-ear utility device 1301a, and any number of tasks may be performed on the smartphone 1314and/or the remote computing facility 1318, according to an embodiment ofthe invention. Tasks that may be most amenable to execution on thesmartphone 1314 and/or the remote computing facility 1318 are tasks thatrequire accessing large databases (e.g., restaurant guides) and/or needa more powerful computing device than can be provided by the in-earutility device 1301 a.

Existing computerized applications can be enabled for operation on, orin conjunction with, the in-ear utility device 1301 a, according to anembodiment of the invention. Thus, a user may be able to accessapplications such as Skype translator, Google translator, WeChat,Facebook message, etc. via the in-ear utility device 1301 a, accordingto an embodiment of the invention. In some embodiments, a version of oneof these existing applications may be tailored for operation on thein-ear utility device 1301 a, e.g., some portion of the applicationresides on the in-ear utility device 1301 a with other application taskshandled remotely. In other embodiments, the in-ear utility device 1301 amay simply engage a remote application.

Tasks that may be amenable to processing outside the in-ear utilitydevice include voice authentication, artificial intelligence, speechrecognition, and real-time translation. However, each of these tasks canalso be performed entirely or partially on the in-ear utility device1301 a. So, for example, the in-ear utility device 1301 a may beconfigured to perform some simple translation tasks while leaving morecomplicated tasks to processing outside the in-ear utility device. Thus,the processor of the in-ear utility device 1301 a may be configured tounderstand which tasks it can perform itself and which tasks requireassistance from another device, according to an embodiment of theinvention.

Similarly, the processor (e.g., the processor 207 shown in FIG. 2A) mayalso be configured for notification response management, according to anembodiment of the invention. So, for example, the in-ear utility device1301 a may be paired with the smartphone 1314. The smartphone 1314 mayhave calendar and/or alarm functions. The smartphone 1314 may not filterits calendar/alarm messages (e.g., “The butcher turns 50 today.”).However, the user of the in-ear utility device 1301 a may not want tohear from the speaker of the in-ear utility device 1301 a everycalendar/alarm message provided by the smartphone 1301 a.

The processor on the in-ear utility device 1301 a may be configured bythe user to play only calendar/alarm messages at or above a particularthreshold, according to an embodiment of the invention. Thecalendar/alarm filter could be provided either on the smartphone 1314and/or on the in-ear utility device 1301 a, according to an embodimentof the invention. The calendar/alarm filter could also be provided by anexternal utility such as Google Calendar. The filter, could, forexample, be an extension to Google Calendar or a similar function.

In operation, for example, the filter instructs the in-ear utilitydevice 1301 a to play only high priority alarm messages. Alternatively,the filter may reside on the smartphone 1314 or remote computingfacility 1318 and simply determine a subset of alarm messages to send tothe in-ear utility device 1301 a, and the in-ear utility device 1301 aplays all the alarm messages of that subset that it receives. So, forexample, “Job interview in 5 minutes” may have the highest priority, andthe platform (e.g., the smartphone 1314 and/or the remote computingfacility 1318) hosting the calendar/alarm filter may send this messageto the in-ear utility device 1301 a for playing to the user while theplatform decides not to send “Send flowers to Joe sometime today” to thein-ear utility device 1301 a such that the user won't hear this messagevia the in-ear utility device 1301 a, according to an embodiment of theinvention.

The filtering function itself may be adjustable by the user and/orautomatically by particular events, according to an embodiment of theinvention. For example, as previously discussed, the in-ear utilitydevice 1301 a may include a driver safety application. If the in-earutility device 1301 a (or a related external system) becomes aware thatthe user is driving an automobile, then the calendar/alarm function mayautomatically engage (or be engaged by an external system in theautomobile itself) to thwart the playing of all calendar/alarm messagesand/or such calendar/alarm messages not at or above a high threshold,according to an embodiment of the invention.

In addition, the processor on the in-ear utility device 1301 a may alsobe configured not to play calendar/alarm messages when the in-earutility device 1301 a is aware that the user is speaking, according toan embodiment of the invention. The in-ear utility device 1301 a maythen schedule replaying of the calendar/alarm message after the passageof a predetermined amount of time, according to an embodiment of theinvention. As previously discussed, the microphones on the in-earutility device 1301 a may be configured to listen to the user's acousticenvironment.

Similarly, as mentioned above, existing applications (e.g., WeChat) maybe enabled for operation on the in-ear utility device 1301 a. Once theseapplications have been enabled, the filtering function described abovemay also be applied to notifications provided by these applications aswell, according to an embodiment of the invention. Thus, the filter inconjunction with the application can determine when, where, and hownotifications from these applications are delivered to the user. Inother words, not all notifications may be provided to the user throughthe speaker of the in-ear utility device 1301 a residing in the user'sear 1305 a, according to an embodiment of the invention.

Various embodiments of the invention have been described in detail withreference to the accompanying drawings. References made to particularexamples and implementations are for illustrative purposes, and are notintended to limit the scope of the invention or the claims.

It should be apparent to those skilled in the art that many moremodifications of the in-ear utility device besides those alreadydescribed are possible without departing from the inventive conceptsherein. The inventive subject matter, therefore, is not to be restrictedexcept by the scope of the appended claims. Moreover, in interpretingboth the specification and the claims, all terms should be interpretedin the broadest possible manner consistent with the context.

Headings and sub-headings provided herein have been provided as anassistance to the reader and are not meant to limit the scope of theinvention disclosed herein. Headings and sub-headings are not intendedto be the sole or exclusive location for the discussion of a particulartopic.

While specific embodiments of the invention have been illustrated anddescribed, it will be clear that the invention is not limited to theseembodiments only. Embodiments of the invention discussed herein may havegenerally implied the use of materials from certain named equipmentmanufacturers; however, the invention may be adapted for use withequipment from other sources and manufacturers. Equipment used inconjunction with the invention may be configured to operate according toconventional protocols (e.g., Wi-Fi) and/or may be configured to operateaccording to specialized protocols. Numerous modifications, changes,variations, substitutions and equivalents will be apparent to thoseskilled in the art without departing from the spirit and scope of theinvention as described in the claims. In general, in the followingclaims, the terms used should not be construed to limit the invention tothe specific embodiments disclosed in the specification, but should beconstrued to include all systems and methods that operate under theclaims set forth hereinbelow. Thus, it is intended that the inventioncovers the modifications and variations of this invention provided theycome within the scope of the appended claims and their equivalents.

It should be noted that while many embodiments of the inventiondescribed herein are drawn to a smart wireless in-ear utility device,various configurations are deemed suitable and may employ variouscomputing devices including servers, interfaces, systems, databases,agents, engines, controllers, or other types of computing devicesoperating individually or collectively. One should appreciate that anyreferenced computing devices comprise a processor configured to executesoftware instructions stored on a tangible, non-transitory computerreadable storage medium (e.g., hard drive, solid state drive, RAM,flash, ROM, etc.). The software instructions preferably configure thecomputing device to provide the roles, responsibilities, or otherfunctionality as discussed below with respect to the disclosed smartin-ear utility device.

All publications herein are incorporated by reference to the same extentas if each individual publication or patent application werespecifically and individually indicated to be incorporated by reference.Where a definition or use of a term in an incorporated reference isinconsistent or contrary to the definition of that term provided herein,the definition of that term provided herein applies and the definitionof that term in the reference does not apply.

As used herein, and unless the context dictates otherwise, the terms“ambient noise” and “ambient sound” have been used synonymously.Similarly, “sound” and “noise” have been used synonymous, except wherethe context shows a difference in meaning, e.g., “meaningful sound frommere noise.” Likewise, “coupled to” is intended to include both directcoupling (in which two elements that are coupled to each other contacteach other) and indirect coupling (in which at least one additionalelement is located between the two elements). Therefore, the terms“coupled to” and “coupled with” are used synonymously. The terms“coupled to” and “coupled with” are also used euphemistically to mean“communicatively coupled with” where two or more networked devices areable to send or receive data over a network.

We claim:
 1. A wireless in-ear utility device, comprising: a body havingat least a portion shaped to fit into a user's ear canal, the bodyhaving a proximal end configured to reside in the user's ear canal at adistance no more than 12 millimeters away from the user's ear drum; amicrophone in the body that receives external sounds from the in-earutility device, wherein the external sounds include sounds produced byan entity other than the user of the in-ear utility device; a datastorage component having an audio profile for the entity; a processorconfigured to perform noise cancellation on the external sounds usingthe audio profile to enhance sounds produced by the entity; and aspeaker located at a distal end of the body that receives the enhancedsounds produced by the entity and plays them to the user of the in-earutility device.
 2. The wireless in-ear utility device of claim 1,further comprising: a wireless communication module fitted into the bodyand configured for wireless communications with at least one externaldevice, wherein the processor provides the enhanced sounds to the atleast one external device via the wireless communications module.
 3. Thewireless in-ear utility device of claim 1, wherein the data storagecomponent has audio profiles for a plurality of entities and wherein theprocessor is further configured to select an audio profile of theplurality of audio profiles for enhancing sounds in the external soundsrepresenting the selected audio profile of the plurality of audioprofiles.
 4. The wireless in-ear utility device of claim 3, furthercomprising: a user-selectable actuator that allows the user to selectthe audio profile from the plurality of audio profiles for theprocessor.
 5. The wireless in-ear utility device of claim 4 wherein theuser-selectable actuator, comprises: an accelerometer located in thebody, the accelerometer configured to detect taps on the user's body andrecord tap data including tap intensity and tap duration, wherein thedetected taps represent an audio profile selection command from thein-ear utility device user, wherein the processor is configured toreceive the tap data from the accelerometer and determine if thereceived tap data matches a predetermined audio profile selectioncommand, wherein the processor applies the selected audio profile whenthe received tap data matches the predetermined audio profile selectioncommand.
 6. The wireless in-ear utility device of claim 1, furthercomprising: a digital signal processor configured to receive theexternal sounds from the microphone and intensify the external soundsbefore sending the external sounds to the processor, wherein theprocessor applies the intensified external sound in enhancing the soundsproduced by the entity.
 7. The wireless in-ear utility device of claim1, further comprising: a wireless communication module fitted into thebody and configured for wireless communications with another wirelessin-ear utility device located in the user's other ear, wherein thewireless communication module receives sounds from the another wirelessin-ear utility device, wherein the processor also applies the soundsfrom the another wireless in-ear utility device in enhancing soundsproduced by the entity.
 8. The wireless in-ear utility device of claim 7wherein the processor improves signal-to-noise ratio of the externalsounds and the external sounds received from the another wireless in-earutility device by performing binaural beamforming to narrow adirectional region within the external sounds and the external soundsfrom the another wireless in-ear utility device so that sounds outsidethe directional region are attenuated.
 9. The wireless in-ear utilitydevice of claim 1, further comprising: an ambient microphone portlocated at the distal end of the body and configured to receive theexternal sounds; and an ambient focused microphone located in the bodythat receives the external sounds via the ambient microphone port,wherein the processor is further configured to include the externalsounds from the ambient microphone in extracting sounds produced by theentity.
 10. The wireless in-ear utility device of claim 1 wherein theentity sound is at least one of a human voice, an animal sound, amachine noise, and a natural sound.
 11. A method for operating wirelessin-ear utility device, comprising: receiving external sounds in amicrophone located in a body of the in-ear utility device, the bodyhaving at least a portion shaped to fit into the user's ear canal andhaving a proximal end configured to reside in the user's ear canal at adistance no more than 12 millimeters away from the user's ear drum,wherein the received external sounds include sounds produced by anentity other than the user of the in-ear utility device; retrieving anaudio profile for the entity from a data storage component attached tothe body; and performing noise cancellation on the external sounds by aprocessor configured to apply the audio profile to enhance soundsproduced by the entity; and playing the enhanced sounds produced by theentity to the user of the in-ear utility device through a speakerlocated at a distal end of the body.
 12. The method of claim 11, furthercomprising: sending the extracted sounds produced by the entity to atleast one external device from a wireless communication module fittedinto the body.
 13. The method of claim 11, wherein the data storagecomponent has audio profiles for a plurality of entities, the methodfurther comprising: selecting by the processor an audio profile of theplurality of audio profiles to enhance sounds in the external soundsrepresented by the selected audio profile of the plurality of audioprofiles.
 14. The method of claim 13, further comprising: receiving theselection of an audio profile for the processor by a user-selectableactuator that allows the user to select the audio profile from theplurality of audio profiles.
 15. The method of claim 14, wherein themethod for the user-selectable actuator comprises: detecting taps on auser's body and recording tap data including tap intensity and tapduration by an accelerometer located in the body; receiving the tap datafrom the accelerometer in the processor, the processor configured todetermine if the received tap data matches a predetermined audio profileselection command; and selecting an audio profile from the data storagecomponent by the processor when the received tap data matches thepredetermined audio profile selection command.
 16. The method of claim11, further comprising: intensifying the external sounds in a digitalsignal processor configured to receive the external sounds from themicrophone and intensify the external sounds before sending the externalsounds to the processor, wherein the processor applies the intensifiedexternal sound in enhancing the sounds produced by the entity.
 17. Themethod of claim 11, further comprising: receiving external sounds fromanother wireless in-ear utility device located in the user's other earvia a wireless communication module fitted into the body; and applyingthe external sounds from the another wireless in-ear utility device bythe processor in enhancing sounds produced by the entity.
 18. The methodof claim 17, further comprising: improving signal-to-noise ratio of theexternal sounds by the processor by performing binaural beamforming tonarrow a directional region of the external sounds and the externalsounds received from the another wireless in-ear utility device so thatsounds outside the region are attenuated.
 19. The method of claim 11,further comprising: receiving ambient sounds in an ambient microphonelocated in the body, wherein the processor is further configured toinclude ambient sounds received from the ambient focused microphone inenhancing the sounds produced by the entity.
 20. The method of claim 11wherein the entity sound is at least one of a human voice, an animalsound, a machine noise, and a natural sound.